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Abstract 

Range  estimation  algorithms  have  been  applied  to  Laser  Detection  and  Ranging 
(LADAR)  data  to  test  for  accuracy  and  precision.  Data  was  acquired  from  Matlab® 
simulations  and  an  experiment  using  the  Advanced  Scientific  Concepts  3-D  flash 
LADAR  camera.  Simulated  LADAR  data  was  based  on  a  Gaussian  pulse  shape 
model  with  Poisson  noise  added.  Simulations  were  performed  to  test  range  estima¬ 
tion  algorithm  performance  with  respect  to  waveform  position  within  the  range  gate. 
The  effectiveness  of  each  algorithm  is  presented  in  terms  of  its  average  root  mean 
square  error  and  standard  deviation  in  1000  trials.  The  measured  data  experiment 
examined  the  effectiveness  of  an  algorithm’s  ability  to  determine  a  range  difference 
between  2  flat  surfaces.  The  algorithms  compared  for  analysis  include  a  peak,  max¬ 
imum  likelihood,  and  matched  filter  estimator.  Various  interpolation  strategies  were 
implemented  in  the  peak  estimator.  The  matched  filter  was  implemented  in  the  time 
and  frequency  domains.  A  normalized  version  of  the  matched  filter  was  also  developed 
and  applied  to  the  LADAR  data.  Three  different  methods  based  on  averaging  were 
developed  to  calibrate  the  pulse  width  of  the  reference  waveform  used  in  the  matched 
Liters  and  maximum  likelihood  algorithms. 

Simulation  results  show  that  a  matched  Liter  produces  a  bias  when  waveforms 
are  oL  center,  but  normalizing  waveforms  before  computing  the  cross  correlation  can 
reduce  the  average  bias  from  0.335  meters  to  0.124  meters.  The  maximum  likelihood 
algorithm  also  produces  a  bias  in  shifted  waveforms,  while  the  peak  estimator  main¬ 
tains  a  nearly  constant  level  of  bias  in  its  measurements  due  to  the  effect  of  shot 
noise  on  waveforms.  In  the  measured  data  sets,  normalization  did  not  reduce  the 
bias  in  measurements  because  it  increased  the  algorithm’s  sensitivity  to  errors  in  the 
reference  waveform  model.  The  maximum  likelihood  algorithm’s  sensitivity  to  errors 
in  modeling  were  revealed  due  to  its  poor  performance  in  the  measured  data. 
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Range  Estimation  Algorithm  Comparison 
in  3-D  Flash  LADAR  Data 


I.  Introduction 

1.1  Problem  Statement 

Laser  Detection  and  Ranging  (LADAR)  systems  have  become  an  integral  part 
of  the  technology  used  by  both  military  and  civilian  personnel  for  increasing  knowl¬ 
edge  of  a  scene.  With  an  expanding  range  of  applications  from  target  recognition  to 
guidance  and  navigation,  there  is  no  shortage  for  research  and  expansion  of  LADAR 
technologies. 

One  fundamental  aspect  of  LADAR  is  understanding  the  ability  to  determine 
the  range  to  an  object  based  on  the  detected  return  pulse  reflected  off  a  target.  The 
goal  of  this  research  is  to  understand  how  different  range  estimation  algorithms  can 
be  applied  to  different  scenarios  based  on  waveform  shape,  size,  and  position  within 
a  respective  range  gate  and  which  estimation  technique  delivers  the  best  performance 
in  terms  of  accuracy,  precision,  and  speed. 

1.2  Research  Goals 

The  primary  goal  of  this  research  is  to  serve  as  a  catalog  comparison  of  range 
estimation  techniques  for  the  Advanced  Scientific  Concepts  (ASC)  3-D  flash  LADAR 
camera  system.  Several  variations  on  algorithms  have  been  tested  in  recent  years, 
but  there  has  not  been  definitive  documentation  of  each  technique,  nor  a  side  by  side 
comparison  of  each  method  under  controlled  conditions. 

This  research  will  also  attempt  to  expand  or  improve  upon  existing  techniques 
in  order  to  increase  speed  and  accuracy.  New  techniques  and  algorithms  discovered 
during  the  research  process  will  also  be  included  in  the  study. 
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Another  goal  of  the  research  is  to  examine  the  performance  of  the  algorithms 
under  hit  mode  and  sular  mode  conditions.  A  description  of  the  camera’s  operating 
modes  is  included  in  Section  2.6.  The  effectiveness  of  a  range  estimation  technique’s 
results  in  hit  mode  will  be  analyzed  by  observing  the  range  estimation  bias  and 
standard  deviation  as  a  function  of  the  position  of  a  return  pulse  waveform  within 
the  range  gate. 

The  final  outcome  of  this  research  will  produce  one  proven  method  for  determin¬ 
ing  the  range  of  a  target  using  the  3-D  flash  LADAR  camera.  It  will  also  be  feasible 
to  describe  when  each  algorithm  will  be  preferred  over  other  algorithms  based  on  the 
noise,  camera  modes,  or  other  distinguishing  factors. 

1.3  Scope  of  Research 

The  research  will  be  performed  on  data  that  is  simulated  via  Matlab®  and  on 
data  produced  from  the  ASC  3-D  flash  LADAR  camera.  While  some  aspects  of  the 
algorithms  will  be  universal  to  any  laser  radar  system,  the  final  product  designed 
will  be  made  with  the  intention  of  its  use  under  the  ASC  3-D  flash  LADAR  camera 
system.  The  hit  mode  operation  study  will  be  specific  to  the  ASC  camera  system 
and  would  only  pertain  to  other  systems  that  may  operate  with  a  similar  threshold 
trigger  technique. 

Noise  introduced  and  simulated  in  the  data  will  be  based  on  the  ASC  camera 
system.  Again,  some  noise  sources  are  universal  to  LADAR  principles,  but  some  may 
be  specific  to  the  camera  system  such  as  specific  dark  current  noise  levels  used  during 
measured  data  observations. 

While  this  research  is  dedicated  to  documenting  and  examining  effectiveness  of 
the  range  estimation  algorithms,  some  additional  sources  of  noise  and  error  may  be 
discovered  along  the  way.  These  noise  sources  will  be  documented  when  observed, 
but  the  optimal  solution  for  their  mitigation  may  not  be  covered  in  the  scope  of  this 
research. 
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1.4  Document  Overview 

The  thesis  document  begins  in  Chapter  1  with  an  introduction  of  the  problem 
and  a  description  of  the  research  performed.  Chapter  2  continues  with  appropriate 
background  information  on  past  work  pertaining  to  the  ASC  3-D  flash  LADAR  camera 
and  the  incorporation  of  LADAR  systems  for  military  applications.  Chapter  2  also 
contains  a  history  of  several  range  estimation  techniques  that  have  previously  been 
applied  to  LADAR  data.  Chapter  3  describes  the  new  developments  related  to  range 
estimation  techniques.  The  methodology  for  performing  the  experiments  in  both 
simulated  and  measured  data  is  included  in  Chapter  4.  Chapter  5  presents  the  results 
of  the  experiments  while  Chapter  6  contains  the  final  conclusions  and  suggestions  for 
future  research  in  related  areas. 
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II.  Background 

Chapter  II  contains  information  within  the  scope  of  the  research  that  will  provide  a 
suitable  background  for  the  reader.  The  background  chapter  contains  information  on 
applications  of  laser  radar,  range  equation  development,  and  range  estimation  theory. 

2.1  Laser  Radar  Development 

Traditional  microwave  radar  systems  operate  in  a  variety  of  wavelengths  within 
the  electromagnetic  spectrum.  Depending  on  the  application,  operation  wavelengths 
can  vary  from  100  m  (High  Frequency  Band)  to  several  millimeters  (Wide  Band)  [6]. 
In  contrast  to  traditional  microwave  radar,  laser  radar  operates  on  a  much  shorter 
wavelength  of  the  electromagnetic  spectrum  typically  ranging  from  10.6  /iin  to  0.4 
pm.  [6].  Light  Detection  and  Ranging  (LIDAR)  systems  are  designed  around  an 
electro-optical  device  that  transmits  an  outgoing  pulse  in  the  optical  electromagnetic 
spectrum.  Receiver  optics  collect  the  return  pulse  data  and  digitize  the  signal  for 
future  use  in  post  processing  algorithms.  Data  can  be  interpreted  to  analyze  the  in¬ 
tensity,  time  of  flight,  and  range  information.  LADAR  systems  are  a  specific  category 
of  LIDAR  where  the  optical  transmission  source  is  a  laser  [6] .  Early  LADAR  systems 
for  3-D  imaging  were  designed  around  scanning  systems  where  a  mechanical  device 
moved  the  beam  across  a  target  area  in  order  to  collect  enough  information  to  build  a 
scene  or  image  of  the  illuminated  area.  By  1996,  scannerless  systems  were  developed 
that  allowed  an  area  with  a  larger  field  of  view  (FOV)  to  be  illuminated  in  a  single 
shot  [10].  Scannerless  systems,  now  often  referred  to  as  flash  systems,  benefit  from 
less  moving  parts  which  can  translate  to  reduced  cost,  smaller  hardware  footprint, 
and  improved  reliability  [10,13].  Avalanche  Photodiode  (APD)  arrays  are  now  the 
customary  technology  used  in  a  Focal  Plane  Array  (FPA)  fashion  to  collect  the  return 
pulse  information  and  form  pixels  of  data  that  can  be  analyzed  using  signal  processing 
techniques  to  determine  the  range  to  each  pixel  [13]. 
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2.2  Laser  Radar  Applications 

LADAR  technology  has  many  diverse  roles  in  both  civilian  and  military  applica¬ 
tions  including  remote  sensing,  target  recognition  and  identification,  and  guidance  or 
navigation  enhancement  [6] .  The  need  for  3-D  imaging  systems  for  military  purposes 
comes  from  an  evolution  of  warfare  that  demands  precision  guided  weapons,  increased 
situational  awareness,  quicker  reaction  times,  and  reduced  collateral  damage.  LADAR 
imaging  can  provide  decision  makers  the  necessary  tools  to  meet  these  demands  both 
on  and  off  the  battlefield.  LADAR  applications  extend  from  land  based  applications 
to  both  aerial  and  maritime  techniques  [3].  One  of  the  most  favored  applications  for 
LADAR  involves  repeatedly  illuminating  an  area  obscured  by  foliage  or  netting  and 
collecting  the  return  data  in  order  to  form  an  image  of  what  is  hidden  beneath  [13]. 
There  is  also  an  emerging  potential  to  use  LADAR  systems  to  allow  helicopter  pi¬ 
lots  see  through  dust  or  sand  clouds  caused  by  spinning  rotors  at  low  altitudes  as 
presented  in  [12].  As  the  demand  for  unmanned  vehicles  increases,  LADAR  systems 
can  be  incorporated  to  provide  vision,  security,  and  collision  avoidance  as  proven 
in  past  trials  with  robotics  [9].  Existing  technology,  such  as  the  Global  Positioning 
System  (GPS),  could  potentially  be  fused  with  LADAR  data  to  further  enhance  the 
information  available  to  decision  makers  and  operators  and  provide  back  up  navi¬ 
gational  controls.  By  accurately  developing  LADAR  technology  and  corresponding 
signal  processing  techniques  required  for  analyzing  the  data,  one  could  improve  target 
recognition,  feature  tracking,  and  other  aspects  of  3-D  imaging  applications. 

2.3  Laser  Radar  Waveform  Models 

In  order  to  accurately  simulate  LADAR  return  pulse  data,  it  is  imperative  to 
have  an  accurate  model  of  the  waveforms  that  are  received  by  the  detector.  A  thorough 
and  robust  development  of  the  waveform  model  will  deliver  more  accurate  results  in 
simulations.  A  well  developed  model  will  also  deliver  more  accurate  range  estimates 
for  algorithms  that  depend  on  a  reference  waveform  such  as  the  matched  filter  and 
maximum  likelihood  algorithms  discussed  in  Section  2.8.  In  general,  the  return  pulse 
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is  formed  as  a  function  of  the  convolution  of  the  outgoing  laser  pulse  and  the  surface 
of  the  target.  The  basic  shape  of  the  original  laser  pulse,  with  the  addition  of  noise 
sources,  should  still  be  maintained.  Three  common  methods  exist  for  modeling  the 
return  pulse  shapes. 

2.3.1  Parabolic  Waveform  Model.  The  simplest  model  is  that  of  an  inverse 
parabolic  waveform.  This  model  has  been  explored  and  tested  in  both  [1]  and  [2],  As 
adapted  from  [2]  the  inverse  parabolic  waveform  is  shown  by 


(2.1) 


where  /(?-&)  is  the  intensity  of  the  pulse  in  photoelectrons  at  range  sample  r^,  G  is 
the  gain  or  amplitude  of  the  pulse,  R  is  the  range  to  the  target,  pw  is  the  half  pulse 
width  of  the  target  in  range  units.  B  represents  the  bias  or  background  noise  of  the 
signal  in  terms  of  photoelectrons,  and  q  represents  the  noise  of  the  sensor  added  to 
the  signal.  As  shown  in  [5]  the  rectangle  function  is  defined  as 

f  1  if  Ixl  <  | 

rect(x )  =  <  "  (2.2) 

y  0  else 

and  is  used  to  limit  the  waveform  to  positive  values. 

Figure  2.1  illustrates  the  idea  that  the  inverse  parabolic  can  provide  a  suitable 
fit  to  a  measured  return  pulse  from  the  ASC  3-D  flash  LADAR. 

2.3.2  Gaussian  Waveform  Model.  Another  option  for  the  waveform  model 
incorporates  a  Gaussian  shaped  pulse  as  the  basis  for  the  waveform  model.  The  width 
of  the  Gaussian  pulse  can  be  adjusted  by  using  different  values  for  the  standard 
deviation.  Based  on  previous  observations  of  measured  data,  the  waveforms  used 
in  simulations  will  be  modeled  as  a  Gaussian  pulse  where  the  intensity,  /(r^),  in 
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Figure  2.1:  a)  Symetric  LADAR  data  with  comparison  to  parabolic  and  Gaussian 
waveform  models,  b)  Skewed  waveform  data  with  a  hybrid  model  used  to 
fit  data.  First  half  of  pulse  is  using  a  2  ns  pulse  width  while  the  second 
half  of  the  pulse  is  using  a  5  ns  pulse  width. 

photoelectrons  of  each  sample  k  is  calculated  using  the  equation 

-( rk-R )2 

IT  +  B  +  q  (2.3) 

y2  7t(Jw 

where  aw  is  the  pulse  width  or  standard  deviation,  G  is  the  amplitude,  rk  is  the  range 
for  a  given  sample  k  in  meters,  and  R  is  the  range  to  the  target.  B  and  q  again 
represent  the  bias  and  noise  added  to  the  signal.  Reference  Figure  2.1  to  observe  the 
fit  the  Gaussian  waveform  model  with  measured  data. 

2.3.3  Hybrid  Waveform  Model.  Another  model  for  simulation  and  building 
a  reference  waveform  involves  the  design  of  a  hybrid  waveform  model.  This  more  com¬ 
plex  method  involves  two  combinations  of  either  the  parabola  and  Gaussian  waveform 
models  with  different  widths  for  each  half  to  yield  a  model  that  is  better  suited  to 
fit  the  measured  return  pulse  from  the  ASC  3-D  flash  LADAR.  Figure  2.1  shows  the 
comparative  fit  of  the  hybrid  and  measured  data.  The  hybrid  waveform  model  is  best 
used  for  waveforms  that  appear  to  be  skewed  or  asymmetric. 
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2-4  Laser  Radar  Range  Equation 


Accurate  range  estimation  techniques  require  a  thorough  understanding  of  the 
principles  behind  the  propagation  of  the  laser  pulses  reflected  off  the  target  in  question. 
A  model  for  the  range  equation  of  the  return  signal’s  intensity  is  presented  in  [6]  and 
will  be  followed  for  the  purposes  of  this  research. 

To  obtain  the  correct  power  in  each  sample  during  simulations,  each  sample  is 
propagated  down  range  and  the  intensity  of  the  return  sample  is  computed  using  the 
range  equation  described  in  [6].  The  range  equation  has  a  direct  dependence  on  the 
area  of  the  target  which  affects  the  cross  section  of  a  target.  The  cross  section  of 
the  target  is  dependent  upon  whether  the  target  is  a  point  source,  Lambertian,  or 
other  type.  In  all  simulations  and  experiments  used  for  this  research  the  target  was 
a  Lambertian  diffuse  target  where  the  cross  section,  cr,  is 


a  =  AptdA 


(2.4) 


where  pt  is  the  reflectivity  of  the  target  and  dA  is  the  area  of  the  target.  The  intensity 
of  the  energy  reflected  from  the  target,  I  target  is  a  function  of  the  cross  section  and 
the  intensity  of  the  laser  illuminating  the  target  shown  by 


j  Efrons  4?r 

targ“  =  MP  *  S; 


(2.5) 


where  Ptrans  is  the  power  transmitted,  R  is  the  range  to  the  target,  and  9t  is  the  beam 
width.  The  power  reflected  from  the  target,  Ptarget ,  is  now  given  by 


Ptarget  &  ^target 


(2.6) 


and  can  be  used  to  find  the  intensity  at  the  aperture,  Iaperture-,  of  the  receiver  using 


*  aperture 


Vatm  Ptarget 
4irfl2 


(2.7) 


where  r]atm  is  the  atmospheric  transmission  factor.  The  intensity  at  the  aperture  is 
next  used  to  calculate  the  power  returned  to  the  receiver,  Psig,  via 

p  laperture^lsys'^D  ^  ^ 

with  the  assumption  that  the  aperture  is  circular  and  D  is  the  diameter  in  meters.  The 
system  efficiency  factor,  r)sys  is  the  transmission  efficiency  of  the  receiver  multiplied 
by  the  efficiency  of  the  detector. 

The  final  end  to  end  range  equation  used  to  measure  each  sample  in  the  wave¬ 
form  is 


p  .  — 

1  sig 


PtransV  sys&VatmD 

1602  R4 


and  is  converted  to  photoelectrons  by  using 


^ sig 


VdQ.P sigG 


hv 


(2.9) 


(2.10) 


where  r/d  is  the  quantum  efficiency,  q  is  the  charge  of  an  electron,  G  is  the  gain  of  the 
LADAR  system,  h  is  Planck’s  constant,  and  v  is  the  frequency  of  the  laser  light. 


2.5  Noise  Sources  in  LADAR  data 

There  are  several  sources  of  noise  in  the  LADAR  data  that  can  lead  to  errors 
in  range  measurements.  This  section  will  describe  natural  sources  of  noise  that  can 
occur  in  the  data.  Sources  of  noise  or  error  that  are  directly  related  to  the  algorithm 
processing  will  be  described  in  the  respective  section  for  each  algorithm. 

2.5.1  Shot  Noise.  Shot  noise  is  used  to  account  for  errors  in  measurements 
as  a  signal  reaches  the  detector  and  is  modeled  via  a  Poisson  distribution  [6].  Shot 
noise  is  related  to  the  strength  of  the  return  signal  and  causes  fluctuations  in  the 
intensity  measurements  that  can  shift  the  peak  of  a  waveform  from  its  true  location. 
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Figure  2.2  shows  the  plot  of  a  simulated  signal  without  shot  noise  and  one  with  shot 
noise  added  that  has  caused  a  shift  in  the  location  of  the  peak  sample. 


Figure  2.2:  Peak  is  easily  shifted  with  addition  of  shot  noise  simulated  via  a  Poisson 
distribution 

2.5.2  Background  Noise.  Because  the  sensor  records  all  light  levels  that 
enter  the  detector,  background  or  ambient  light  levels  can  have  an  effect  on  the 
intensity  of  the  return  signal  [6] .  The  background  light  levels  are  added  as  an  averaged 
bias  term  that  if  known,  could  be  subtracted  from  the  data  based  on  the  waveform 
models  previously  described.  The  background  for  a  data  collection  scenario  can  be 
obtained  by  measuring  data  received  by  the  optics  without  actually  firing  the  laser 
pulse.  The  background  intensity  during  a  data  capture  is  shown  in  Figure  2.3.  Light 
levels  can  fluctuate  between  laser  shots  so  this  calibration  step  would  need  to  be 
performed  for  each  data  collection. 

2.5.3  Dark  Current.  Dark  current  is  false  positive  readings  of  electrons 
by  the  detector  that  can  add  noise  to  the  signal  [6].  All  electro-optic  sensors  are 
subject  to  dark  current.  Current  leakage  varies  from  system  to  system  based  on  the 
internal  physics  and  temperature.  The  dark  current  of  the  system  can  be  estimated 
by  measuring  return  data  when  the  system’s  receiver  optics  are  covered.  The  average 
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Figure  2.3:  Example  of  background  radiation  noise  compared  to  signal  reflected  from 
target  in  3-D  flash  LADAR  camera  data 

values  for  a  dark  current  measurement  in  the  ASC  3-D  flash  LADAR  camera  have 
been  obtained  and  can  be  used  to  remove  dark  current  bias  from  the  measured  data.  If 
dark  current  values  are  used  they  could  be  included  into  the  overall  bias  (B)  term  that 
includes  the  background  noise.  Figure  2.4  illustrates  the  capture  of  dark  current  levels 
obtained  by  covering  the  lens  and  recording  the  waveforms.  The  data  was  obtained 
while  the  camera  was  in  hit  mode,  so  the  spike  in  levels  is  an  anomaly  caused  by  the 
threshold  trigger.  Only  samples  7-19  would  be  averaged  to  obtained  the  value  of  the 
dark  current  intensity. 

2.5.4  Timing  Jitter.  Timing  jitter  is  another  source  of  noise  caused  by 
small  fluctuations  in  the  internal  clock  of  the  system.  Very  small  changes  in  timing 
can  cause  changes  in  the  time  the  pulse  is  received  and  can  lead  to  errors  in  range 
measurements.  A  method  for  mitigating  the  errors  caused  by  timing  jitter  in  the 
measured  data  experiment  is  presented  in  Section  4.3. 
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Figure  2.4:  Dark  current  readings  from  ASC  3-D  flash  LADAR  camera.  Spike  in  first 
7  samples  is  anomaly  caused  by  use  of  hit  mode.  Samples  have  been  sorted 
and  marker  slice  was  removed.  Final  12  samples  would  be  averaged  to 
obtain  an  indication  of  dark  current  levels  in  camera  system. 


2.6  ASC  Camera  Description 

The  ASC  3-D  flash  LADAR  camera  operates  by  emitting  a  laser  pulse  which  is 
reflected  off  a  target.  The  reflected  energy  is  processed  by  the  camera’s  receiver  optics. 
Data  is  digitized  from  the  optical  sensors  and  transferred  to  a  computer  for  further 
processing  of  the  data.  The  receiver  is  built  around  an  array  of  128  x  128  APDs.  The 
sampling  rate  of  the  system  is  420  MHz.  The  memory  storage  capacity  allows  for  20 
samples  to  be  collected  and  stored  in  hard  memory.  Each  sample  represents  a  distance 
of  approximately  36  cm,  so  the  20  samples  collectively  allow  for  a  range  gate  that  is 
slightly  over  7.1  meters.  However,  one  sample  serves  as  a  marker  slice  which  is  removed 
from  the  data  and  stands  as  a  reference  for  range  measurements.  This  19  sample  set 
of  data  builds  a  usable  range  gate  with  a  depth  of  about  6.8  meters.  The  ASC  3-D 
flash  LADAR  camera  has  the  ability  to  operate  in  two  different  modes,  sular  and  hit 
mode.  Hit  mode  may  also  be  referred  to  as  stop  mode  in  some  documentation  [11], 
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2. 6. 1  Sular  Mode  Description.  In  sular  mode,  the  user  is  allowed  to  specify  a 
range  gate  with  the  depth  previously  described.  Only  buffered  return  pulse  data  that 
falls  within  the  specified  range  gate  is  stored  to  form  the  20  samples  returned  to  the 
user  for  post  processing.  When  the  approximate  or  exact  range  to  the  target  is  known, 
sular  mode  is  a  suitable  option  to  ensure  that  waveforms  are  properly  captured  by 
the  system.  However,  if  the  range  to  the  target  is  unknown  or  if  there  is  a  question  of 
whether  or  not  a  target  is  present,  hit  mode  may  serve  as  a  more  appropriate  choice. 

2.6.2  Hit  Mode  Description.  There  are  conflicting  views  on  the  hit  mode 
operations  in  the  ASC  3-D  flash  LADAR  camera.  Based  on  previous  camera  designs, 
it  was  believed  that  hit  mode  operations  performed  based  on  a  user  defined  threshold 
in  terms  of  photoelectron  counts.  Return  pulse  data  is  buffered  through  the  system 
until  the  threshold  of  photoelectrons  is  reached  [11].  When  the  threshold  is  crossed, 
8  samples  before  the  threshold  are  stored,  and  12  samples  after  the  threshold  are  also 
stored.  Again,  this  presents  the  20  samples  used  for  range  calculations.  1  sample 
serves  as  a  marker  slice  and  is  removed  from  the  set  doing  post  processing.  In  order 
to  reference  the  waveforms  of  each  pixel  properly  and  build  the  128  x  128  different 
range  gates,  a  128  x  128  array  is  stored  in  the  camera’s  memory  that  contains  the 
range  at  which  the  threshold  was  crossed  for  each  pixel  of  data.  This  is  referred  to  as 
the  hit  mode  buffer  and  allows  the  system  to  build  a  separate  range  gate  for  each  of 
the  128  x  128  waveforms  captured  by  the  system. 

Upon  further  examination  of  hit  mode  operations  in  the  newer  camera  used 
to  collect  data  for  this  research,  it  appears  that  the  previous  hit  mode  operation 
assumptions  are  not  correct  or  do  not  hold  true  for  the  new  camera  system.  The  128  x 
128  hit  mode  buffer  did  not  contain  values  that  could  sensibly  be  range  measurements 
to  the  threshold  crossing.  Every  other  row  of  values  contains  zeros,  which  leads  to  the 
assumption  there  is  most  likely  an  error  in  the  camera’s  system  when  creating  the  hit 
mode  buffer.  For  non  zero  values,  it  would  appear  that  they  represent  a  marker  that 
dictates  which  range  gate  the  corresponding  waveform  would  use.  Due  to  the  current 
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errors  in  the  hit  mode  buffer,  only  about  half  the  pixels  collected  in  hit  mode  can  be 
properly  used  with  the  range  estimation  algorithms.  After  determining  the  proper 
range  gate  to  match  to  the  waveforms,  they  would  not  be  processed  any  differently 
from  sular  mode  data.  Because  of  the  complications  associated  with  the  new  cameras 
hit  mode  buffer,  all  the  measured  data  analyzed  when  comparing  the  performance  of 
the  range  estimation  algorithms  was  collected  in  sular  mode. 

2. 7  Gain  Variation  in  the  ASC  3-D  flash  LADAR  System 

It  has  been  shown  that  the  gain  of  the  ASC  3-D  flash  LADAR  system  used 
in  previous  research  is  not  constant  as  expected,  but  varies  nonlinearly  with  respect 
to  time  [11],  As  presented  in  [7]  and  [11],  the  gain  of  the  APD  array  can  fluctuate 
depending  on  the  internal  voltage  regulator  and  current  distribution.  Return  pulses 
from  the  LADAR  camera  would  ideally  be  the  same  shape  and  size  when  reflected 
from  the  same  target,  but  through  experimentation  described  in  [7]  and  [11]  it  has 
been  shown  the  waveforms  fluctuate  as  seen  in  Figure  2.5.  The  waveforms  in  Figure 
2.5  are  the  collected  return  signals  when  the  target  was  a  155  cm  x  155  cm  square 
plywood  board  placed  approximately  70  meters  from  the  receiver  and  perpendicular 
to  the  line  of  sight. 

As  shown  in  Figure  2.5  the  fluctuation  in  gain  causes  the  shape  of  the  return 
pulses  to  change.  The  shape  of  waveforms  can  become  skewed  or  warped  which 
makes  it  impractical  to  create  an  accurate  waveform  model  based  on  the  skewed 
waveforms  [7] .  The  lack  of  a  consistent  shape  that  can  be  easily  modeled  can  mislead 
certain  range  algorithms  such  as  the  matched  filter  technique  described  in  Section 
2.8.2.  Similar  to  the  effect  of  shot  noise,  gain  variation  could  possibly  shift  the  peak 
of  a  waveform  by  several  samples,  which  could  lead  to  errors  in  range  measurements 
when  using  certain  range  estimation  techniques.  Although  it  was  not  tested  in  the 
scope  of  this  research,  it  is  possible  that  algorithms  such  as  a  leading  edge  detector 
may  still  properly  process  the  unequalized  signals. 
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Figure  2.5:  Unequalized  return  pulses  from  a  target  approximately  70  meters  from 
receiver  and  perpendicular  to  line  of  sight.  Pulses  are  not  symmetric,  it 
appears  the  second  half  of  the  pulses  are  wider  than  the  first  half. 

In  order  to  accurately  compare  the  performance  of  the  range  estimation  algo¬ 
rithms  and  design  a  model  for  the  waveforms,  the  gain  variation  should  be  taken  into 
account  and  corrected  before  further  steps  are  performed. 

One  area  of  concern,  however,  is  the  effect  of  the  gain  variation  when  the  camera 
is  operating  in  hit  mode.  In  hit  mode  the  APDs  operate  independently  and  each  could 
be  bring  at  different  times  based  on  the  threshold  set  by  the  operator.  In  this  situation, 
some  of  the  critical  assumptions  and  calculations  based  on  the  background  levels  and 
relationships  will  not  necessarily  hold  true.  The  current  draw  that  causes  the  gain  to 
fluctuate  will  not  be  as  strong  if  the  individual  APDs  are  not  operating  at  the  same 
time.  Section  2.7.1  will  describe  in  detail  the  method  proposed  in  [7]  for  reducing  the 
effects  of  gain  variation  on  range  estimation  error. 

2.7.1  Estimation  and  Equalization  of  Gain  Variation.  In  this  section,  a 
technique  for  estimating  the  gain  variation  is  presented.  The  application  of  the  equal¬ 
ization  technique  is  based  on  the  findings  of  [7]  and  [11].  Unless  otherwise  noted,  all 
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equations  and  references  for  calculating  the  gain  are  under  the  assumption  that  the 
camera  is  operating  in  sular  mode. 

The  variation  in  gain  is  a  direct  result  of  the  change  in  load  during  the  time  of 
data  collection  when  the  voltage  regulator  cannot  maintain  a  constant  output  [11], 
As  each  sample  is  taken,  the  responsiveness  of  the  APD  will  vary  and  the  gain  will 
fluctuate.  As  seen  in  Figure  2.6,  the  background  signal  power  in  a  non-target  pixel 
appears  constant  and  then  drops  significantly  at  about  sample  6  or  7  to  coincide 
with  the  the  first  reception  of  return  pulses  reflected  from  the  target.  In  theory, 
the  background  light  levels  observed  in  pixels  that  do  not  contain  target  information 
should  remain  constant.  The  relationship  between  the  background  power  and  gain 
given  in  [7]  is 

is(t)  =  ^ Pb  (2.11) 

riv 

where  G  is  the  gain,  q  is  the  charge  of  an  electron,  rjq  is  the  quantum  efficiency,  h  is 
Planck’s  constant,  and  v  is  the  frequency  of  the  laser  light.  To  evaluate  the  gain,  the 
following  equation  for  the  model  of  data,  d(rk),  is 

d(rk)  =  G(rfc)^  (2.12) 

*5 avg 

where  G(r k)  is  the  gain  for  sample  rk,  and  B}Tk'>  is  the  ratio  of  background  in  an 
individual  pixel,  B(rk),  to  the  average  background  value  in  the  pixel  over  the  19 
samples,  Bavg  [7].  If  this  equation  is  rearranged  in  order  to  solve  for  the  gain,  G(rk), 
then  it  would  appear  as 

G(rk)  =  d{rk)^^  (2.13) 

The  data,  d(rk),  has  already  been  acquired  and  the  average  background  values,  Bavg , 
can  also  be  calculated  from  the  previous  collection  of  background  noise.  To  estimate 
the  background  in  the  actual  data  that  is  affected  by  the  gain,  B(rk),  a  cluster  of 
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pixels  that  did  not  contain  any  target  data  during  a  shot  that  included  a  target  in  the 
scene  was  used  as  the  B(rk )  values.  These  target  free  pixels  were  pixels  that  while 
illuminated  by  the  laser  did  not  receive  any  return  pulse  data.  In  the  data  set  used 
for  the  gain  variation  studies  these  pixels  can  generally  be  found  in  the  upper  right 
or  upper  left  corner  of  the  scene.  The  illuminated  scene  created  by  the  return  pulse 
data  can  be  seen  in  Figure  2.7.  The  background  readings  would  be  affected  by  the 
gain  the  same  as  the  other  pixels.  The  average  background  over  the  same  pixels  was 
taken  from  a  background  data  capture  that  was  recorded  separately  as  seen  in  Figure 
2.3.  The  resulting  value  was  used  as  an  estimate  of  the  gain.  This  method  creates  an 
array  of  gain  values  that  is  a  function  of  each  sample.  An  example  of  the  estimated 
average  gain  value  for  19  samples  when  the  target  was  perpendicular  to  the  line  of 
sight  is  shown  in  Figure  2.8. 

Equalization  of  the  data  is  accomplished  by  dividing  the  measured  data  wave¬ 
forms  by  the  estimated  gain  waveform  on  a  sample  by  sample  basis.  Figure  2.9  shows 
the  results  of  applying  the  equalization  method  to  the  raw  waveforms.  The  leading 
edge  of  the  waveforms  is  much  smoother,  but  the  trailing  edge  is  still  somewhat  dis¬ 
torted.  It  can  also  be  seen  that  the  equalized  waveforms  allow  the  sample’s  intensities 
to  continue  to  climb  upward  in  value,  while  in  the  unequalized  waveforms  they  are 
cut  short  before  actually  reaching  the  true  location  of  the  target  sample.  Waveforms 
still  appear  to  be  a  combination  of  two  pulse  widths  that  would  benefit  from  a  hybrid 
waveform  model. 

In  [7]  it  is  shown  that  the  gain  equalization  technique  does  not  eliminate  all  of 
the  error  in  range  measurements,  but  does  reduce  the  error.  Further  experimentation 
is  necessary  before  a  strong  conclusion  can  be  made  regarding  the  performance  of  the 
gain  estimation  technique. 

2.1.2  Additional  Notes  on  Gain  Variation.  The  data  collected  and  used 
during  this  research  was  gathered  with  a  newer  version  of  the  ASC  3-D  flash  LADAR 
camera.  While  the  new  camera  still  has  some  characteristics  of  the  other  cameras  and 
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Figure  2.6:  Unequalized  return  pulse  in  pixel  that  contains  target  data  compared  to 
an  averaged  background  pixel  that  does  not  contain  target  data.  Note 
the  drop  in  background  level  that  corresponds  to  the  return  of  the  signal 
reflected  off  target. 

operates  in  the  same  fashion,  it  may  be  difficult  to  compare  the  data  and  equalization 
technique  of  the  older  camera  systems  to  the  newer  version.  Also,  a  new  technique 
for  eliminating  gain  variation  involving  optical  filters  has  been  implemented  for  the 
purpose  of  this  research.  This  method  will  be  described  in  Section  3.1.  Because  of 
the  apparent  lack  of  gain  variation  in  the  new  camera  and  optical  filter  technique 
employed,  the  gain  estimation  technique  described  in  [7]  was  not  applied  to  the  data 
used  in  the  range  estimation  experiments  conducted  during  the  scope  of  this  research. 
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Figure  2.7:  Image  of  scene  formed  from  LADAR  intensity  data  viewed  at  sample 
10.  Target  is  about  70  meters  from  receiver  and  normal  to  line  of  sight. 
White  square  indicates  region  of  scene  used  to  compute  background  levels 
in  target  scene  for  gain  estimation. 


Figure  2.8:  Average  gain  values  when  target  was  normal  to  line  of  sight.  The  gain 
drops  significantly  at  the  samples  that  correspond  to  the  first  return  pulses 
from  the  target. 
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Figure  2.9:  Equalized  waveforms  from  the  same  pixels  shown  in  Figure  2.5. 
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2.8  Range  Estimation  Theory 

This  section  will  detail  current  range  estimation  techniques  used  in  processing 
LADAR  data  and  provide  a  derivation  or  description  of  each. 

2.8. 1  Peak  Estimation.  One  of  the  simplest  methods  of  estimating  the  range 
of  a  return  pulse  from  a  LADAR  system  is  the  peak  estimator.  Based  on  the  creation 
of  the  waveforms,  the  largest  power  of  the  return  sample  will  theoretically  coincide 
with  the  sample  corresponding  to  the  target  range.  A  critical  element  of  using  the 
peak  estimator  is  having  a  waveform  that  includes  the  true  peak  of  the  return  pulse. 
The  peak  of  the  received  waveform  is  recorded,  and  the  corresponding  range  value 
from  that  particular  sample  is  chosen  as  the  range  estimate.  As  shown  in  Section  5.2, 
the  peak  estimator  algorithm  is  highly  effective  in  simulations  when  there  is  no  noise 
present,  but  in  realistic  conditions  with  the  presence  of  shot  noise  the  peak  estimator 
can  easily  be  misled  as  illustrated  by  Figure  2.2.  The  peak  estimator  can  be  described 
by  the  equation 


Rpeak  =  argma xd(rk)  (2.14) 

rk 

where  d(rk)  is  the  waveform  currently  used  for  range  estimation.  Rpeak  is  the  range 
value  returned  by  the  peak  estimator.  To  achieve  sub-sample  range  estimations, 
the  waveform  can  be  interpolated  using  various  strategies  before  searching  for  the 
peak.  However,  in  the  presence  of  noise,  interpolation  could  lead  to  more  error  in 
range  estimation.  Figure  2.10  shows  how  interpolation  can  change  the  data  used  in 
the  peak  estimator.  The  results  in  Chapter  V  will  show  the  benefits  of  an  accurate 
interpolation  strategy  when  using  the  peak  estimator. 

2.8.2  Matched  Filter.  The  matched  filter  evolves  from  the  idea  of  using  a 
shifting  reference  model  waveform  to  cross  correlate  with  the  data  waveform.  The 
shifted  waveform  with  the  highest  correlation  is  used  to  determine  the  correct  range 
estimate.  It  is  essential  to  have  an  accurate  model  of  the  waveform  when  using  a 
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(a)  Uninterpolated  Samples 


(b)  Linear  Interpolation 
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(c)  Cubic  Spline  Interpolation  (d)  Shape  Preserving  Cubic  Inter¬ 
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Figure  2.10:  a)  Simulated  LADAR  pulse  with  no  interpolation  between  two  samples. 

b)  Linear  interpolation  used  to  create  additional  samples,  c)  Cubic  Spline 
Interpolation,  d)  Shape  preserving  (of  original  sampled  waveform)  cubic 
interpolation. 


matched  filter,  particularly  when  determining  the  width  of  the  waveform.  A  discussion 
of  the  sensitivity  of  the  range  estimates  with  respect  to  the  shape  and  width  of  the 
reference  model  waveform  is  presented  in  Chapter  V.  There  are  several  variations  on 
the  matched  filter  algorithms.  The  largest  variation  depends  on  whether  or  not  the 
algorithm  is  implemented  in  the  time  or  frequency  domain.  Both  methods  were  used 
in  the  scope  of  this  research  and  are  presented  in  the  following  sections. 

2.8.2. 1  Time  Domain  Correlation.  The  matched  filter  is  implemented 
by  comparing  the  data  waveform  to  a  reference  waveform  using  a  cross  correlation 
technique.  In  [8]  this  is  shown  in  two  dimensions  as  a  measure  of  distance  between 
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the  two  functions  described  by 


d2u(u,  v )  =  ^[/(x,  y)  —  t(x  —  u,y  —  v )]2  (2.15) 

x,y 

which  can  be  simplified  to  one  dimension  for  application  to  the  LADAR  waveforms. 
In  one  dimension  this  is  shown  as 

d),t(u)  =  (2.16) 

X 

where  a  change  in  variable  nomenclature  leads  to 

N 

cd,f(T)  =  X^(rfe)  ~~  f(rk  “  T^2  (2‘17) 

k= 1 

where  c^j(t)  is  the  correlation  value  between  the  two  functions  d(rk )  and  f(rk )  at 
a  shift  value  of  r.  d(rfc)  is  the  LADAR  return  pulse  data  waveform  at  range  sample 
rk  and  f(rk )  is  the  reference  waveform  based  off  of  a  predetermined  model  at  range 
sample  rk.  The  reference  waveform  is  created  such  that  the  center  or  peak  value  is  at 
the  shift  value  of  r.  N  is  the  total  number  samples  in  the  data  waveform.  The  next 
step  in  evaluating  the  cross  correlation  is  expanding  the  terms  that  are  a  function  of 
the  square  value.  This  leads  to  the  equation 

N 

c|/(r)  =  ~  2 d(rk)f(rk  -  r)  +  f2(rk  -  r)]  (2.18) 

k= 1 

which  can  be  examined  to  note  that  two  of  the  terms,  d2(rk )  and  f2(rk  —  r)  would 
remain  constant  after  passing  through  the  summation  process.  Terms  that  do  not 
change  as  a  function  of  r  will  be  removed  from  the  equation  because  they  do  not 
affect  the  location  of  maxima  and  minima  points.  This  simplification  step  leaves 

N 

4/(r)  =  Yy~2d(<rk^(drk  ~ r)]  (2-19) 

k=  1 
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which  can  also  be  simplified  to  remove  the  negative  and  scalar  factors  that  do  not 
have  any  effect  on  changes  with  respect  to  r.  This  final  expression  is  shown  as 

N 

Cdj(r)  =  ^[d(rfc)/(rfe  ~  t)]  (2.20) 

k= 1 

which  describes  the  cross  correlation,  C,  of  d(rk)  and  /(r^)  with  respect  to  the  shift 
value  of  r. 

The  reference  waveform,  /(r*,),  is  generated  using  either  the  Gaussian,  inverse 
parabola,  hybrid,  or  other  model  with  a  peak  at  different  locations  corresponding  to 
the  guess  at  the  range  of  the  target  (r).  The  data  waveform  and  the  reference  wave¬ 
form  samples  are  individually  multiplied  by  one  another  and  the  resulting  products 
summed  to  form  a  correlation  value  for  that  particular  shift.  The  range  estimated 
by  the  matched  filter  in  the  time  domain,  Rmftd,  corresponds  to  the  value  of  r  that 
maximizes  the  correlation,  C(t)  shown  by 

Rmftd  =  Rgate  (arg max C'(t))  (2.21) 

where  Rgate  is  the  range  gate  of  all  potential  range  values  tested  by  the  shifting 
reference  wave.  The  correlation  process  in  the  time  domain  is  illustrated  in  Figure 
2.11. 

In  [8]  it  is  mentioned  that  when  two  signals  with  energies  that  can  vary  with 
position  are  used  in  the  matched  filter  technique,  the  algorithm  will  fail.  This  is 
illustrated  in  Section  5.2  where  it  is  shown  that  there  is  a  bias  in  the  algorithm  when 
the  data  waveforms  are  not  centered  in  the  range  gate.  A  method  to  reduce  the  bias 
by  normalizing  the  energy  of  the  signals  is  proposed  in  Section  3.2. 

2. 8. 2. 2  Frequency  Domain  Correlation.  The  matched  filter  technique 
can  also  be  implemented  in  the  frequency  domain  by  taking  advantage  of  the  correla¬ 
tion  property  of  the  Fourier  Transform  described  in  [4] .  Given  the  Fourier  Transform 
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Figure  2.11:  Figures  a)-c)  illustrate  the  reference  waveform  as  it  shifts  from  left  to 
right  across  the  range  gate.  Figures  d)-e)  illustrate  the  correlation  value 
as  the  reference  waveform  shifts  position. 


of  the  data  is  D(w)  and  the  Fourier  Transform  of  the  reference  waveform  is  F(w)  then 

iV-1 

=  ^{^2d(rk)f{rk-r)}  (2.22) 

n= 0 

which  can  be  approximated  as 


J?{C(t)}  =  D(u)F(u) 


(2.23) 


From  here,  the  range  is  found  by  finding  the  maximum  value  of  the  function  JF-{C(t)} 
and  hireling  its  corresponding  range  sample.  This  can  lead  to  a  much  faster  compu¬ 
tation  as  the  number  of  operations  is  drastically  reduced.  Sub  sample  interpolation 
is  achieved  by  zero  padding  both  the  data  and  reference  waveform  in  the  frequency 
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domain  before  multiplication.  The  desired  level  of  interpolation  is  achieved  by  first 
finding  the  current  sampling  frequency,  A/,  by  using 


A/ 


1 

NiAti 


(2.24) 


where  N\  is  the  current  number  of  samples  and  Ati  is  the  current  time  between 
samples.  If  the  desired  level  of  time  between  samples  is  At2  then  the  number  of 
samples  to  be  added  by  zero  padding,  N2,  is  found  by  the  equation 


N2 


1 

A hKf 


N, 


At  i 

A T2 


(2.25) 


N2  can  now  be  used  with  Ni  to  determine  the  appropriate  number  of  zeros  needed  to 
add  to  either  side  of  the  waveform.  If  a  desired  number  of  samples  is  required  instead 
of  the  preset  sampling  distance,  interpolation  can  be  accomplished  in  the  same  manner 
by  solving  for  the  appropriate  variable,  ffowever,  it  would  not  be  possible  to  properly 
normalize  a  waveform  as  described  in  Section  3.2  if  this  method  were  used. 


2.8.3  Maximum  Likelihood.  The  maximum  likelihood  (ML)  estimator  is 
derived  by  determining  the  probability  density  function  (PDF)  of  the  data  at  one 
range  sample  instance,  rk.  In  [2]  it  is  determined  that  the  largest  source  of  noise 
would  come  from  the  shot  noise  in  the  signal.  Because  shot  noise  is  modeled  as 
Poisson,  the  PDF  for  one  sample  is 

P[D(rk)  =  d{rk)}  =  1  k,d{rk)] -  (2.26) 

where  I(rk)  is  the  data  generated  by  one  of  the  waveform  models  described  in  Section 
2.3.  The  data  is  designated  by  d(rk).  Due  to  the  assumed  independence  between  the 
noisy  samples,  the  joint  PDF  is  the  product  of  the  individual  PDFs  shown  by 


P[D(rk)  =  d{rk)Vke[l,N]]  = 


n 


/(rfe)dhfc)e--fhD 

d(rk)\ 


(2.27) 


26 


To  simplify  the  process  of  maximizing  the  probability,  the  maximum  of  the  log  of 
Equation  (2.27)  will  be  found.  The  first  step  is  accomplished  by  taking  the  natural 
log  of  both  sides  shown  by 


/  -A  /(Vi')d(rfe)e_/(rfe) 

In  ( P  l D(rk )  =  d(rk)Vk  e  [1,  N]])  =  In  (  1 


which  is  simplified  to 


^  V  d(rkV- 


k=  1 


This  expression  can  further  be  simplified  to 


N 


L  =  Y l  In  (/(rfc)d(rfc)e“J(rfc))  -  In  {d(rk)\) 


k= 1 


and 

N 

L  =  ^ln  (I(rk)d(rk))  +  In  (e“/(rfc))  -  Zn  (d(rfc)!) 

fc=i 

The  previous  equation  simplifies  to  eliminate  the  exponential  term  by 

N 

L  =  Y,ln  (I(rk)d^)  -  I(rk )  -  Zn  (d(rk)\) 

k=  1 


(2.28) 


(2.29) 


(2.30) 


(2.31) 


(2.32) 


The  term  ln(d(rk)\)  will  be  a  constant  as  a  function  of  the  range  parameter  after 
the  summation  and  only  serves  as  a  bias  to  the  overall  likelihood  calculation.  Bias 
terms  do  not  affect  the  location  of  maxima  or  minima  points  and  can  be  dropped 
from  further  expressions.  This  simplification  leaves  the  log  likelihood  equation  as 

N 

L  =  ^Zn(/(rfc)d^)  -I(rk)  (2.33) 

k=  1 
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At  this  point,  it  is  acceptable  to  substitute  in  the  expression  for  the  model  chosen 
for  the  intensity  of  the  waveform,  /(r*).  For  the  purpose  of  this  derivation,  it  will 
be  assumed  that  Gaussian  waveform  model  shown  in  Equation  (2.3)  and  described 
in  Section  2.3.2  is  chosen  as  the  reference  model.  Making  the  substitution  for  /(r^) 
yields  the  equation 


N 

L(G ,  R ,  aw)  =  In 
k=  1 


G  -hk-R? 
— - — — e  2o™ 
'\Z‘2/k(jw 


(2.34) 


The  likelihood  function,  L,  has  now  been  annotated  to  show  that  the  estimate  is  a 
function  of  the  the  amplitude  of  the  signal,  G,  the  true  range  to  the  target,  i?,  and  the 
standard  deviation  or  pulse  width  of  the  Gaussian  reference  model,  aw.  A  separate 
method  based  on  curve  fitting  to  estimate  aw  will  be  executed  before  the  maximum 
likelihood  method  is  performed  and  is  described  in  Section  3.3.  Obtaining  an  estimate 
for  <tw  will  leave  two  unknowns  at  this  point  in  the  process,  R  and  G. 

To  estimate  G,  the  derivative  of  the  likelihood  function  in  Equation  (2.34)  will 
be  found  and  maximized  with  respect  to  G.  This  step  yields  an  estimation  of  G  as 


G 


Ef=i  d{rk) 


Yn  _ - _ e 

Z-^k—1  y/2naw 


(2.35) 


which  can  be  placed  back  into  the  waveform  model,  /(r^).  Now,  substituting  the 
equation  for  /(r^)  with  the  estimate  for  G  into  the  log  likelihood  function,  L(G,R), 
yields  an  equation  that  is  no  longer  a  function  of  G  and  R ,  simply  just  R. 

The  closed  form  solution  for  R  that  maximizes  L{R)  is  difficult  to  obtain,  so 
it  is  found  by  iteratively  trying  different  values  of  range  samples  until  the  maximum 
value  of  L(R)  is  found  by 


Rail  =  Rgate  ( arg  max  L(R) 
\  R 


(2.36) 


(a)  LADAR  Return  Pulse 


x  io4 


Range  Value  (m) 


(b)  Log  Likelihood  Curve 


Figure  2.12:  a)  LADAR  signal  used  in  maximum  likelihood  range  estimation  algo¬ 
rithm.  b)  Curve  produced  by  using  maximum  likelihood  algorithm. 
Maximum  value  corresponds  to  range  sample  returned  as  estimate  by 
algorithm. 


where  Rml  is  the  range  estimated  by  the  maximum  likelihood  algorithm.  Sub-sample 
range  estimation  is  achieved  by  trying  a  smaller  step  size  between  the  range  sample 
estimates  in  question.  Figure  2.12  shows  an  example  of  the  log  likelihood  plot,  with 
the  peak  corresponding  to  the  correct  range  sample. 
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III.  New  Techniques  in  Range  Estimation 

Chapter  III  describes  a  new  technique  for  estimating  the  range  of  an  object  using 
laser  radar,  and  methods  for  developing  a  suitable  waveform  model  for  the  shape  of 
the  laser  pulse.  An  alternative  method  for  reducing  gain  variation  is  also  presented 
in  Chapter  III. 

3.1  Alternative  Method  for  Reducing  Gain  Variation 

In  addition  to  the  method  tested  in  [7],  other  ideas  have  been  proposed  to  reduce 
the  effects  of  gain  variation.  In  the  data  collected  for  this  research,  an  optical  filter 
was  placed  over  the  lens  of  the  receiver  optics  to  reduce  the  amount  of  light  collected 
by  the  system.  The  filter  was  a  12  nanometer  wide  bandpass  filter  with  a  center 
of  1.55  micrometers.  The  filter  reduced  the  total  amount  of  radiation  entering  the 
detector  and  reduced  or  nearly  eliminated  the  effects  of  the  gain  variation.  This  is 
shown  by  the  return  pulse  shapes  that  appear  to  be  only  affected  by  the  shot  noise, 
clock  noise,  and  timing  jitter  of  the  system.  Figure  3.1  shows  an  example  of  some 
waveforms  collected  from  a  target  approximately  4.5  meters  away  with  an  optical 
filter  in  place  compared  to  data  from  the  previous  camera  system  without  the  use  of 
an  optical  filter. 

3.2  Normalized  Matched  Filter  Algorithm 

As  described  in  Section  2.8.2  and  shown  in  Section  5.2  the  matched  filter  range 
estimation  algorithm  can  produce  a  bias  in  waveforms  that  are  not  in  the  center  of 
the  range  gate.  A  proposed  method  to  reduce  or  eliminate  this  bias  is  to  normalize 
the  energy  in  each  signal  using  the  correlation  coefficient. 

If  the  original  data  waveform  is  d(rk),  then  the  normalized  signal,  d(rk),  is 
defined  as 

d{rk)  -  d{rk) 
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Figure  3.1:  a)  Waveforms  exhibit  gain  variation  which  skews  return  pulse  shape  when 
no  filter  is  used.  b)LADAR  return  pulses  from  camera  with  use  of  optical 
filter  to  reduce  overall  intensity  and  eliminate  gain  variation;  pulses  are 
nearly  symmetric 


where  d(rk)  is  the  mean  value  of  the  data  and  a d  is  the  standard  deviation  of  the  data 
waveform  defined  by 


CTd 


Eh,  (d(n)  -  din))2 


N 


(3.2) 


where  N  is  the  total  number  of  samples  in  the  waveform.  The  reference  waveform, 
/(rfc),  is  normalized  in  the  same  method  and  annotated  by 

/>*)  =  f(n)  ~  fin)  (3.3) 

°7 

and 

o ~ f  =  hM  -  JR T  (3,4) 

where  /(r^)  is  the  normalized  reference  waveform,  /(r^)  is  the  mean  value  of  the 
reference  waveform,  and  Of  is  the  standard  deviation  of  the  reference  waveform  data. 
From  here,  the  range  is  estimated  the  same  as  the  time  domain  matched  filter  pre¬ 
viously  described,  but  now  the  normalized  signals  are  used  in  the  calculation  of  the 


31 


cross  correlation,  Cdj{r),  as  shown  in 


N 

Cdj(r)  =  Y^[d(rk)Krk  -  r)]  (3.5) 

k=  1 

The  range  is  estimated  by  finding  the  range  sample  that  maximizes  Equation  (3.5)  as 
described  in 


Rnctd  =  Rgate  (arg max C(r)j  (3.6) 

where  Rnctd  is  the  range  to  the  target  estimated  by  the  normalized  matched  filter 
algorithm  performed  in  the  time  domain. 

The  normalized  matched  filter  technique  can  also  be  applied  by  selectively  choos¬ 
ing  pixels  that  only  show  a  desired  degree  of  correlation.  This  selective  process  is 
designed  with  the  intent  of  discarding  waveforms  that  do  not  match  the  model  and 
could  deliver  poor  range  estimates.  The  selective  normalized  matched  filter  algorithm 
could  potentially  help  with  the  sensitivity  of  the  algorithm  to  deviations  from  the  ref¬ 
erence  model.  This  is  practical  in  research  purposes  when  pixel  by  pixel  estimations 
are  performed  to  analyze  the  system  and  algorithms,  but  in  reality  all  pixels  of  data 
may  be  needed  depending  on  the  particular  application. 

3.3  Reference  Waveform  Design 

Both  the  matched  filter  and  the  maximum  likelihood  algorithms  rely  heavily  on 
the  assumption  that  the  shape  of  the  return  pulse  is  known.  Even  after  the  Gaussian 
or  inverse  parabola  model  is  chosen,  the  algorithms  must  still  know  the  correct  width 
of  the  pulse  in  order  to  effectively  choose  the  range.  This  section  will  present  the 
techniques  used  to  build  a  reference  waveform. 

3.3.1  Pulse  Width  Estimation  Techniques.  Each  of  the  algorithms  presented 
in  this  chapter  use  the  same  fundamental  process  to  estimate  the  width  of  the  wave- 


32 


form.  The  pulse  width  estimation  (PWE)  technique  works  by  first  choosing  range  of 
widths  to  guess  at.  A  reference  waveform  is  created  with  each  guess  and  centered  on 
the  pulse  in  question.  The  error  is  determined  by  subtracting  the  two  pulses,  squaring 
the  error,  and  finding  the  mean  value.  The  next  value  for  the  pulse  width  is  tried  and 
the  process  is  repeated.  This  process  is  also  repeated  with  the  reference  waveform 
centered  1-2  samples  off  center  in  case  the  peak  of  the  waveform  is  shifted  due  to  the 
effects  of  shot  noise.  Each  mean  square  error  value  is  recorded  and  produces  a  curve 
that  is  used  to  determine  the  minimum  error.  The  pulse  width  guess  that  produced 
the  least  error  is  chosen  as  the  estimate  by  the  PWE. 

Based  on  the  visual  observation  of  the  return  pulses  from  the  LADAR  camera 
that  did  not  exhibit  strong  gain  variation  and  skewed  pulses,  a  Gaussian  return  pulse 
was  used  as  the  basis  for  the  model.  It  appeared  as  a  better  fit  to  due  to  the  inclusion 
of  tails  in  the  model.  The  inverse  parabolic  method  did  not  include  this  trait.  The  use 
of  the  optical  filter  produced  symmetric  waveforms,  so  the  hybrid  waveform  model 
was  not  chosen.  An  experiment  was  conducted  to  determine  which  of  the  pulse  width 
estimation  techniques  would  perform  the  best  by  using  simulated  data  to  compare 
the  results  of  each  technique.  The  benefit  of  using  simulated  data  in  this  case  is  that 
the  pulse  width  is  known  and  can  be  used  to  determine  if  there  is  any  bias  in  the 
different  techniques. 

At  this  point  in  the  process  of  building  the  reference  waveform,  several  options 
are  present  to  choose  from.  The  following  techniques  were  developed  and  explored 
during  the  scope  of  this  research: 

3. 3. 1.1  Averaging  Pulses  then  Estimating.  The  Averaging  Pulses  then 
Estimating  (APE)  algorithm  averages  LADAR  return  pulse  waveforms  and  makes  one 
estimate  to  use  for  all  the  waveforms  based  on  results  of  the  averaged  waveform.  The 
APE  method  is  beneficial  because  only  one  estimation  is  made  for  a  large  group  of 
pixels  which  results  in  a  faster  algorithm.  However,  as  illustrated  by  Figure  3.2,  aver¬ 
aging  different  return  pulses  could  result  in  an  averaged  pulse  that  is  wider  than  the 
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individual  returns  the  estimate  is  designed  for.  Target  surfaces,  range  differences,  tim¬ 
ing  jitter,  or  clock  error  can  all  cause  pulse  widths  to  vary  slightly  and  shift  position. 
However,  if  the  pulses  are  properly  aligned  then  the  averaging  is  beneficial  because 
the  random  noise  can  be  averaged  out,  resulting  in  a  more  accurate  estimation. 

3.3. 1.2  Estimating  Widths  then  Averaging.  The  Estimating  Widths 
then  Averaging  (EWA)  algorithm  estimates  the  individual  pulse  width  of  each  wave¬ 
form  in  a  group  of  pixels.  The  reference  waveform  is  built  using  the  mean  value  of 
the  estimations.  EWA  is  computationally  slower  than  APE,  but  could  be  more  accu¬ 
rate  because  it  does  not  depend  on  the  averaged  waveform  that  could  be  artificially 
widened  due  to  sources  of  noise  previously  described. 

3.3. 1.3  Estimating  Widths  No  Averaging.  The  Estimating  Widths  No 
Averaging  (EWNA)  algorithm  estimates  the  individual  pulse  width  of  each  waveform 
in  a  group  of  pixels  the  same  as  the  EWA  algorithm.  The  difference  between  EWA 
and  EWNA  is  that  a  separate  reference  waveform  is  built  for  each  pixel  based  off 
the  estimated  pulse  widths  stored  in  memory.  The  EWA  and  EWNA  should  com¬ 
putationally  take  the  same  amount  of  time,  but  the  EWNA  algorithm  requires  more 
memory.  The  results  in  Section  5.1  will  show  that  for  the  purposes  of  this  research 
there  is  not  much  benefit  to  using  EWNA  versus  EWA.  In  fact,  due  to  error  caused 
by  noise  in  the  LADAR  return  pulses  EWNA  may  actually  be  less  beneficial  when 
creating  a  reference  model  because  none  of  the  noise  is  averaged  out  of  the  pulse  width 
estimates. 


34 


Sample  Number 

Figure  3.2:  LADAR  return  pulses  in  different  pixels  that  when  averaged  form  a  pulse 
that  is  slightly  wider  than  the  individual  returns. 
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IV.  Methodology 

Chapter  IV  documents  the  design  and  setup  of  different  experiments  used  to  test  the 
techniques  described  in  Chapters  II  and  III.  Included  in  this  chapter  is  an  additional 
explanation  of  the  experiment  used  to  analyze  the  effectiveness  of  the  PWE  tech¬ 
niques.  The  design  and  setup  of  the  simulated  and  measured  LADAR  return  pulse 
collection  experiments  is  also  presented  in  Chapter  IV. 

4-1  Pulse  Width  Estimation  Experiment  Design 

To  evaluate  the  PWE  technique  described  in  Section  3.3  a  simple  experiment 
was  designed  to  estimate  pulse  widths  in  simulated  data.  The  data  was  created  in 
a  similar  way  to  that  described  in  Section  4.2.  Instead  of  varying  the  amplitude  or 
position  of  the  waveforms,  a  set  of  Gaussian  shaped  reference  waveforms  were  created 
with  varying  pulse  widths  ranging  from  1  ns  to  6  ns  in  0.5  ns  increments.  Figure  4.1 
shows  an  example  of  several  of  the  pulse  widths  used  in  the  experiment. 

The  raw  PWE  technique  was  applied  to  the  set  of  noiseless  waveforms  and  the 
results  were  recorded.  Next,  a  3-D  cube  of  23  x  23  pixels  was  made  to  create  529 
waveforms.  These  values  were  chosen  because  it  was  similar  to  the  amount  of  pixels 
available  for  range  estimation  in  the  measured  data  set.  Poisson  noise  was  added  to 
simulate  the  shot  noise  created  under  realistic  conditions.  This  cube  of  data  was  used 
to  evaluate  the  APE,  EWA,  and  EWNA  algorithms  discussed  in  Section  3.3.  The 
results  were  recorded  and  are  presented  in  Section  5.1. 

4-2  Simulated  Data  Experiment  Design 

Before  the  range  estimation  algorithms  were  applied  to  measured  LADAR  data, 
a  simulation  of  data  was  performed.  The  goal  of  the  simulation  was  to  examine  the 
performance  of  the  range  estimation  algorithms  under  hit  mode  conditions.  In  order 
to  create  an  environment  for  the  experiment  to  be  successful,  it  would  be  necessary 
to  generate  waveforms  that  were  at  different  positions  within  the  range  gate.  This 
section  will  describe  the  conditions  of  that  experiment. 
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Sample  Sample  Sample 


(a)  2  ns  pulse  width  (b)  4  ns  pulse  width  (c)  6  ns  pulse  width 

Figure  4.1:  Examples  of  simulated  LADAR  waveforms  of  varying  widths  used  for 
PWE  tests. 

In  the  simulation  the  target  was  placed  at  100  meters  from  the  receiver.  The 
outgoing  pulse  was  modeled  as  a  Gaussian  waveform.  The  range  equation  described 
in  Section  2.4  was  used  to  calculate  the  number  of  photoelectrons  received  by  the 
system.  Shot  noise  was  added  to  the  waveforms  using  the  Poisson  distribution  model. 
The  values  chosen  for  the  parameters  of  the  range  equation  are  shown  in  Table  4.1. 
Table  4.1:  Parameter  Values  for  Simulated  Data 


Parameter  Values 

P trans 

1  mJ 

Detector  Size 

1  ft  m 

e 

0.05  radians 

Solar  Irradiance 

1  kW  /.un/m2 

Focal  Length 

100  mm 

System  Bandwidth 

500  MHz 

D 

10  mm 

Pd 

0.3 

A 

1.55  //m 

1 latm 

1 

G 

20 

pt 

0.10 

R 

100  m 

q 

1.602  xl0~19 coulombs 

h 

6.626  xlO-34  J  s 

Data  samples  were  simulated  from  80  to  120  meters.  Figure  4.2  shows  the  sample 
buffered  waveform  without  noise  before  any  hit  mode  specifications  are  made  to  it. 
Instead  of  setting  a  threshold  and  allowing  waveforms  to  be  cropped  to  achieve  the 
desired  positions  within  the  range  gate,  a  20  sample  window  was  used  to  shift  through 
the  main  wave  pulse  to  collect  the  waveforms.  Waveforms  could  be  positioned  on  the 
left,  center,  or  right  side  of  the  range  gate.  In  addition,  waveforms  at  the  edges  of  the 
range  gate  were  allowed  to  shift  outside  of  the  range  gate  in  order  to  clip  the  amplitude 
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Figure  4.2:  Simulated  noiseless  data  buffered  through  LADAR  sytem 

before  the  true  peak  of  the  original  waveform  was  reached.  A  total  of  20  waveforms 
were  created  and  used  as  the  basis  for  the  noisy  waveform  range  estimations.  Figure 
4.3  shows  an  example  of  several  of  the  waveforms  created  for  the  simulation. 

After  creating  the  20  waveforms,  the  next  step  was  to  add  simulated  shot  noise 
to  each  waveform.  After  the  random  noise  was  added,  the  range  to  the  target  was 
estimated  using  each  of  the  range  estimation  techniques.  The  process  of  adding  noise 
and  estimating  the  range  was  repeated  1000  times  for  each  waveform. 

The  sampling  rate  of  the  simulated  system  allows  for  the  collection  of  a  data 
point  at  about  every  60  cm.  As  shown  in  Figure  4.4  the  true  peak  of  the  waveform 
should  be  at  100  m,  but  due  to  the  sampling  rate  the  closest  samples  to  the  peak  are 
contained  at  99.8  and  100.4  meters.  Without  some  form  of  sub-sample  interpolation, 
these  are  the  closest  values  a  range  estimation  technique  without  interpolation  will 
find. 

In  addition  to  testing  the  various  range  estimation  algorithms,  the  PWE  algo¬ 
rithms  (APE,  EWA,  and  EWNA)  were  also  tested  in  conjunction  with  the  estimation 
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(a)  Left  Side  of  Range  Gate  (b)  Center  of  Range  Gate  (c)  Right  Side  of  Range  Gate 

Figure  4.3:  Examples  of  waveforms  created  in  various  positions  in  the  range  gate  used 
in  the  simulated  experiment. 


techniques  that  depend  on  the  standard  deviation.  The  true  pulse  width  was  known 
so  algorithms  were  also  tested  using  the  known  value  for  the  pulse  width.  Several  dif¬ 
ferent  interpolation  strategies  were  also  tested  with  the  peak  estimator  in  simulated 
data  to  determine  if  better  results  could  be  obtained  in  the  presence  of  the  simulated 
shot  noise.  The  results  of  the  simulated  experiment  are  presented  in  Section  5.2. 


4-3  Measured  Data  Experiment  Design 

Measured  data  was  collected  from  the  ASC  3-D  flash  LADAR  camera  in  Novem¬ 
ber  2008  at  Wright-Patterson  AFB  using  equipment  provided  by  the  Air  Force  Re¬ 
search  Laboratory  (AFRL)  Sensors  Directorate  Electro-Optics  Division  (RYJ).  This 
section  will  describe  the  conditions  and  the  design  of  that  experiment. 

The  camera  and  laser  were  attached  to  a  platform  approximately  4.25  -  4.88 
meters  from  two  target  boards.  The  target  boards  were  were  made  of  a  styrofoam 
poster  board  material  about  0.635  cm  thick  and  were  not  coated  with  any  additional 
materials  to  affect  the  reflectivity  of  the  targets.  The  target  boards  were  placed  be¬ 
tween  1.22  -  1.83  meters  apart  from  one  another  and  kept  perpendicular  to  the  line 
of  sight  of  the  LADAR  system.  The  two  target  boards  were  spaced  horizontally  in 
a  fashion  that  would  allow  the  edge  of  the  near  board  to  overlap  with  the  far  board 
and  allow  both  to  be  seen  in  the  FOV  of  the  LADAR  camera.  Figure  4.5  is  used  to 
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Figure  4.4:  Comparison  of  sampled  waveform  to  original  model  where  peak  value  is 
not  sampled  and  illustrates  need  for  sub-sample  considerations  in  range 
estimation  algorithms. 

illustrate  the  set  up  of  the  measured  data  collection.  The  design  of  the  experiment 
allowed  a  range  difference  between  the  two  boards  to  be  determined  from  the  range 
estimation  algorithms  instead  of  an  absolute  range.  The  method  of  examining  range 
differences  versus  absolute  range  was  chosen  because  using  a  range  difference  mea¬ 
surement  eliminates  the  timing  jitter  noise  and  allows  measurements  from  different 
clock  cycles  to  be  compared  to  one  another.  For  example,  if  the  model  of  the  data  at 
one  range  instance,  dn(r),  with  the  timing  jitter  noise  added  to  the  waveform  is 

dn(r)  =  d(r)  +  ntj  (4.1) 

where  d(r)  is  timing  jitter  noise  free  data,  and  ntj  is  the  timing  jitter  noise,  then 
subtracting  two  measurements,  dnl(r)  and  dn2(r ),  from  the  same  cube  of  data  where 
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the  timing  jitter  is  the  same  would  yield 


dni{r) 


dniir) 

=  di(r) 

dn2{r ) 

=  d2 (r ) 

■  dn2(r ) 

=  di(r ) 

A  D(r) 

=  dx(r) 

A  D(r) 

=  d\(r) 

+  ntj 
+  ntj 

+  ntj  -  ( d2(r )  +  ntj) 
+  ntj  -  d2{r)  -  ntj 
-  d2(r ) 


(4.2) 


where  A D(r)  is  a  range  difference  measurement  between  two  pixels. 

100  pulses  were  fired  from  the  camera  in  order  to  collect  enough  data  to  attempt 
to  average  out  the  noise  over  the  multiple  frames  of  data.  Each  pulse  creates  a  3- 
dimensional  cube  of  data  made  with  the  pixel  location  corresponding  to  the  y-  and 
x-  axes  and  the  photoelectron  intensities  of  the  waveforms  registering  along  the  range 
axis. 

The  target  boards  were  placed  at  different  distances  to  attempt  to  create  wave¬ 
forms  that  would  be  captured  either  on  the  left  side,  center,  or  right  side  of  the  range 
gate  to  recreate  the  conditions  of  experiment  performed  with  simulated  data. 

Table  4.2  lists  the  data  hies  collected  and  the  difference  in  range  between  the 
two  boards  that  would  serve  as  a  truth  measurement  when  testing  for  bias  in  the 
range  estimation  algorithms.  The  hies  were  saved  in  a  .SEQ  format,  which  could  be 

Table  4.2:  File  names  for  measured  data  collection  experiment  and  range  difference 
truth  values. 


File  Name 

Range  Difference  (m) 

Sular9ftl.SEQ 

1.1938 

Sular9ft2.SEQ 

1.4732 

Sularl9ftl.SEQ 

1.3462 

Sularl9ft2.SEQ 

1.8542 
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Figure  4.5:  Illustration  of  measured  data  collection  set  up. 

read  into  Matlab®.  Section  4.3.1  will  describe  the  methods  used  to  process  the  data 
and  prepare  it  for  the  range  estimation  algorithms. 

4-3.1  Processing  of  .SEQ  files.  The  data  collected  from  the  camera  was 
exported  to  an  attached  laptop  and  stored  for  further  use.  Data  contained  in  the 
.SEQ  hies  was  read  into  Matlab®  via  functions  provided  by  AFRL/RYJ  personnel. 
The  data  extracted  from  a  .SEQ  hie  was  organized  into  data  structures  for  easy 
navigation  through  the  information.  Each  data  cube  corresponds  to  a  pulse  fired 
from  the  laser  and  is  stored  as  a  frame  of  data.  The  samples  stored  in  each  cube  of 
waveforms  are  initially  placed  out  of  order  based  on  the  location  of  a  marker  slice  as 
shown  in  Figure  4.6.  The  sample  immediately  following  the  marker  slice  corresponds 
to  the  location  of  the  first  sample  of  data.  Information  contained  in  the  marker  slice 
is  not  useful  to  range  estimations  and  is  removed  from  the  sample  set  thus  reducing 
the  overall  number  of  usable  samples  per  waveform  to  19. 

After  sorting  the  cubes  of  data  and  removing  the  marker  slice,  the  next  step  was 
to  process  each  cube  and  apply  the  range  estimation  algorithms.  In  order  to  build  a 
reference  waveform  for  the  matched  filters  and  maximum  likelihood  algorithms,  the 
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Figure  4.6:  Example  of  a  LADAR  pulse  that  has  not  yet  been  sorted.  Minimum, 
zeroed  out,  value  is  good  indication  of  location  of  marker  slice  in  data. 

pulse  widths  of  the  waveforms  were  estimated  using  the  methods  described  in  Section 
3.3.  In  the  first  two  sets  of  data  collected,  the  noise  levels  were  too  high  in  the  first 
10  samples  and  overpowered  the  rest  of  the  waveform.  These  10  noisy  samples  were 
removed  from  the  data  set.  In  the  second  two  sets  of  data,  where  the  closest  target 
board  was  moved  further  away  from  the  camera,  the  first  sample  was  too  noisy  and 
was  removed  from  the  data  set.  Figure  4.7  shows  a  waveform  that  has  been  sorted 
based  on  the  marker  slice  and  contains  overly  noisy  samples.  As  the  target  boards 
were  moved  further  away,  waveforms  without  the  initial  noisy  samples  were  obtained, 
leading  one  to  believe  there  is  a  minimum  range  at  which  the  ASC  3-D  flash  LADAR 
system  can  operate. 

The  first  two  hies  were  taken  with  the  camera  approximately  4.25  meters  from 
the  first  board.  Based  on  the  fact  that  the  first  10  samples  were  always  dominated  by 
background  noise,  it  appears  that  the  camera  has  a  minimum  range  of  approximately 
3.5  meters.  This  was  not  further  tested  during  the  experiments  and  is  just  a  side 
observation  that  could  prove  useful  in  future  tests.  When  the  target  boards  were 
moved  further  away,  the  noisy  samples  were  reduced  to  1  sample  that  had  to  be 
removed  from  the  waveforms.  Figure  4.8  shows  a  set  of  intensity  images  revealing  the 
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of  LADAR  return  pulse  that  after  being  sorted  based  on  marker 
first  samples  exhibit  a  high  degree  of  background  noise.  Possible 
of  a  minimum  range  for  detection  by  ASC  3-D  flash  LADAR 

two  target  boards.  The  near  board  is  seen  in  sample  slice  3,  and  the  far  board  is  seen 
in  sample  slice  8.  The  information  shown  in  Figures  4.8  -  4.10  is  taken  from  data 
set  Sularl9ft2.SEQ  and  provides  a  fair  representation  of  the  other  data  sets.  Figure 
4.9  shows  a  set  of  waveforms  from  row  80  to  give  an  example  of  the  variation  among 
waveforms  used  in  range  estimation.  Figure  4.10  shows  how  the  difference  in  range 
between  the  two  target  boards  can  be  roughly  estimated  by  looking  at  the  difference 
in  samples  between  the  waveforms. 

4-3.2  Application  of  Range  Estimation  Algorithms.  In  total,  16  different 
variations  of  the  range  estimation  algorithms  were  applied  to  the  measured  data  set. 
The  peak  estimator  was  applied  with  no  interpolation,  linear  interpolation,  cubic 
spline  interpolation,  and  shape  preserving  spline  interpolation  of  the  waveform.  The 
matched  filter  was  implemented  in  the  time  and  frequency  domain  using  the  three 
different  PWE  techniques.  The  three  PWE  techniques  were  also  applied  to  both 
the  maximum  likelihood  estimator  and  normalized  matched  filter  algorithm.  The  16 
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Pixel  Number  Pixel  Number 

(a)  Near  Board  Intensity  Image  (b)  Far  Board  Intensity  Image 

Figure  4.8:  Intensity  images  from  set  of  data  when  target  boards  were  1.85  meters 
apart.  Near  board  image  in  (a)  is  from  cube  slice  3  and  far  board  image 
in  (b)  is  from  cube  slice  8. 

combinations  of  range  estimation  were  applied  to  each  of  the  4  sets  of  data  using  an 
area  of  approximately  121  pixels  and  then  500  pixels.  The  larger  region  of  pixels  was 
used  to  determine  if  branching  out  to  the  noisier  pixels  at  the  edge  of  the  camera’s 
field  of  view  would  affect  the  range  estimation  results.  The  results  of  the  experiments 
are  presented  in  Section  5.3. 
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Figure  4.9:  Waveforms  taken  from  pixel  row  80  looking  at  data  from  various  columns 
in  both  the  near  and  far  board  returns. 


Figure  4.10:  Comparison  of  waveforms  from  near  board  versus  far  board.  Actual 
distance  between  target  is  1.85  meters  or  about  5.18  samples 
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4-4  Methods  for  Analyzing  Results 


This  section  describes  the  processes  and  metrics  used  to  determine  the  effective¬ 
ness  of  the  range  estimation  techniques. 

4-4-1  Simulated  Data.  In  the  simulated  data,  the  true  range  to  the  target 
was  100  meters.  To  test  for  bias  in  range  measurements,  the  Root  Mean  Square 
Error  (RMSE)  in  each  of  the  20  waveform  positions  for  an  estimation  technique  was 
computed  using 


(4.3) 


where  R  is  the  true  location  of  the  target,  Rn  is  the  estimated  range  found  by  the 
algorithm  in  noisy  waveform  n,  and  N  is  the  total  number  of  measurements  (1000). 
The  RMSE  is  in  units  of  meters. 

To  test  for  the  stability  of  the  estimation  techniques,  the  standard  deviation  in 
the  measurements  was  also  found.  Standard  deviation  was  calculated  using 


(4.4) 


where  r  is  the  mean  value  of  the  estimates.  The  oerror  is  in  units  of  meters. 

In  total,  20  RMSE  and  aerror  calculations  were  found  for  each  range  estimation 
algorithm.  This  would  allow  the  error  to  be  plotted  as  a  function  of  waveform  position 
within  the  range  gate.  In  addition  to  these  two  calculations  of  error  as  a  function  of 
waveform  position,  the  average  RMSE  and  standard  deviation  was  also  calculated  for 
waveforms  in  the  center  and  edge  of  the  range  gate. 

The  total  error  was  found  by  treating  the  RMSE  and  aerror  as  two  independent 
random  variables  whose  sum  could  be  found  by 


Errortot  =  sjRMSE 2  +  a\ 


error 


■2 


(4.5) 
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where  Errortot  is  in  units  of  meters. 


4-4-2  Measured  Data.  In  the  data  collected  from  the  ASC  3-D  flash  LADAR 
camera,  a  similar  method  for  finding  the  RMSE  and  variance  is  used  to  analyze  the 
results.  The  RMSE  and  crerror  were  calculated  for  each  range  estimation  technique 
using  Equations  (4.3)  and  (4.4).  In  the  measured  data  sets,  range  differences  between 
the  two  target  boards  were  used  as  the  basis  for  the  range  measurement.  For  each  of 
the  100  cubes  of  data,  the  range  difference  was  found  by  subtracting  the  distance  to 
each  pixel  in  the  far  board  from  the  distance  to  each  pixel  in  the  near  board.  The 
mean  value  of  the  range  difference  was  then  used  as  the  range  difference  value  for  a 
particular  cube.  The  mean  value  could  be  plotted  as  a  function  of  the  cube  number  or 
laser  pulse  time.  This  analysis,  presented  in  Chapter  V  would  show  the  measurements 
as  a  function  of  time  and  allow  the  user  to  see  if  there  was  any  indication  of  a  trend 
in  the  clock  based  on  the  results.  Among  these  average  values,  the  RMSE  error 
was  found  and  the  standard  deviation  was  also  calculated.  The  total  error  was  also 
calculated  using  Equation  (4.5). 

Results  were  also  analyzed  by  examining  differences  in  the  results  based  on  the 
number  of  pixels  used  in  the  calculated  the  estimation.  Tests  were  run  with  either 
approximately  121  or  500  pixels.  In  addition  the  results  were  also  analyzed  based  on 
which  PWE  algorithm  was  used  as  described  in  Section  3.3. 
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V.  Results 


Chapter  V  contains  the  results  of  the  simulated  and  measured  data  results. 


5.1  Pulse  Width  Estimation  Experiment  Results 

5.1.1  PWE  Results  with  Noiseless  Data.  To  test  for  bias  in  the  PWE 
technique,  the  PWE  algorithm  was  first  applied  to  a  set  of  waveforms  that  had  no  noise 
added.  With  the  exception  of  the  narrowest  pulse  widths,  the  estimator  performed 
perfectly.  Table  5.1  shows  the  true  and  estimated  values  of  the  PWE  algorithm  under 
noiseless  scenario.  The  average  error  in  the  experiment  was  0.023  ns.  The  error  in 
the  narrowest  pulse  widths  is  due  to  undersampling  of  the  signal  which  misled  the 
algorithm.  Based  on  the  results  obtained  in  this  experiment  it  is  feasible  to  assume 
the  technique  is  unbiased  without  noise  present.  The  experiment  was  continued  by 
testing  the  PWE  technique  on  waveforms  with  noise  added. 

Table  5.1:  Results  of  noiseless  PWE  experiment  show  the  lack  of  bias  in  the  estimator. 


Actual  Pulse  Width  (ns) 

Est.  Pulse  Width  (ns) 

Error  (ns) 

1.0 

1.2 

0.2 

1.5 

1.6 

0.1 

2.0 

2.0 

0 

2.5 

2.5 

0 

3.0 

3.0 

0 

3.5 

3.5 

0 

4.0 

4.0 

0 

4.5 

4.5 

0 

5.0 

5.0 

0 

5.5 

5.5 

0 

6.0 

6.0 

0 

5.1.2  PWE  Results  with  Noisy  Data.  The  next  step  was  to  conduct  the 
same  experiment  but  with  Poisson  distributed  noise  added  to  the  waveforms.  529 
noisy  waveforms  were  used  to  simulate  a  23  x  23  array  of  pixels.  The  PWE  estimation 
technique  was  applied  via  the  APE  and  EWA  methods  to  determine  if  there  were  any 
conclusions  that  could  be  drawn  regarding  the  performance  of  the  PWE  algorithm 
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in  a  noisy  environment.  In  the  APE  algorithm  experiment,  the  waveforms  were  first 
averaged  and  then  used  to  End  the  pulse  width.  Table  5.2  shows  the  results  of  the 
PWE  experiment  using  the  APE  algorithm.  The  results  were  the  same  as  when  the 
PWE  technique  was  applied  to  noiseless  waveforms.  This  shows  that  the  Poisson 
distributed  noise  can  be  mitigated  from  waveforms  by  averaging  them.  However,  if 
the  waveforms  have  shifted  positions  due  to  range  differences  or  timing  jitter  then 
averaging  could  produce  a  waveform  that  is  wider  than  the  original  pulse  as  shown  in 
Figure  3.2.  To  test  the  EWA  method,  the  individual  pulse  width  was  calculated  for 


Table  5.2:  Results  of  PWE  experiment  when  using  APE  algorithm. 


Actual  Pulse  Width  (ns) 

Est.  Pulse  Width  (ns) 

Error  (ns) 

1.0 

1.2 

0.2 

1.5 

1.6 

0.1 

2.0 

2.0 

0 

2.5 

2.5 

0 

3.0 

3.0 

0 

3.5 

3.5 

0 

4.0 

4.0 

0 

4.5 

4.5 

0 

5.0 

5.0 

0 

5.5 

5.5 

0 

6.0 

6.0 

0 

each  simulated  pixel’s  waveform.  The  mean  value  of  the  pulse  width  estimate  over  the 
529  pixels  was  recorded  as  the  EWA  estimate.  Table  5.3  lists  the  results  of  the  EWA 
experiment.  The  average  error  was  0.1545  ns  which  is  an  order  of  magnitude  greater 
than  the  APE  method.  There  is  also  a  noticeable  increase  in  the  standard  deviation 
of  the  pulse  width  estimates  as  the  pulse  width  increases.  In  the  measured  data, 
however,  it  may  prove  more  effective  to  to  implement  the  EWA  or  EWNA  method 
when  determining  the  range  due  to  the  slight  shifts  in  waveforms  that  could  occur. 

5.2  Range  Estimation  Simulated  Experiment  Results 

The  simulated  experiment  described  in  Chapter  IV  was  performed  with  both 
noiseless  and  noisy  waveforms.  The  results  are  presented  in  the  following  sections. 
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Table  5.3:  EWA  results 


Actual  Pulse  Width  (ns) 

Avg.  Est.  Pulse  Width  (ns) 

Error  (ns) 

a  (ns) 

1.0 

1.280 

0.280 

0.0535 

1.5 

1.641 

0.141 

0.0804 

2.0 

2.107 

0.107 

0.1295 

2.5 

2.633 

0.133 

0.1990 

3.0 

3.124 

0.124 

0.2635 

3.5 

3.673 

0.173 

0.3314 

4.0 

4.213 

0.213 

0.4071 

4.5 

4.686 

0.186 

0.4469 

5.0 

5.168 

0.168 

0.5201 

5.5 

5.635 

0.135 

0.5725 

6.0 

6.040 

0.040 

0.6428 

5.2.1  Results  of  Simulation  Without  Noise.  To  test  for  any  natural  bias 
in  the  range  estimation  algorithms,  the  simulation  was  conducted  first  on  noiseless 
waveforms.  Different  interpolation  methods  were  applied  to  the  peak  estimator  algo¬ 
rithm.  The  pulse  width  dependent  waveforms  used  the  EWNA  pulse  width  estimation 
technique  as  well  as  the  known  pulse  width  value.  The  other  PWE  methods  were  not 
applied  because  they  would  be  irrelevant  without  noise  added.  Because  noise  was  not 
added  to  the  signal,  only  one  trial  was  performed  for  each  waveform.  Only  the  RMSE 
was  calculated,  because  the  standard  deviation  is  undefined  for  one  measurement. 

Table  5.4  lists  the  average  RMSE  of  the  range  estimation  algorithms  at  both 
the  center  10  samples  and  the  10  samples  on  the  left  and  right  side  of  the  range  gate. 
The  only  unbiased  estimators  are  the  peak  estimator  using  the  spline  method  and 
the  normalized  matched  filter.  The  matched  filter  in  the  time  domain  and  maximum 
likelihood  show  a  low  average  RMSE  (1  mm  and  3  mm  respectively)  in  the  center  of 
the  range  gate.  In  contrast,  for  waveforms  shifted  off  center  the  matched  filter  in  the 
time  domain  produced  an  average  RMSE  of  0.310  m  while  the  maximum  likelihood 
algorithm  produced  a  slightly  higher  average  RMSE  of  0.315  m.  The  frequency  domain 
implementation  of  the  matched  filter  has  an  average  RMSE  of  0.225  m  in  the  center 
and  0.297  at  the  edges  of  the  range  gate.  As  seen  in  Figure  5.1  the  frequency  domain 
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implementation  of  the  matched  filter  is  not  as  sensitive  in  the  bias  caused  by  off  center 
waveforms,  but  still  has  an  overall  bias  in  even  the  centered  waveforms. 

Table  5.4:  Results  of  simulation  of  range  estimation  when  no  noise  is  added  to  wave¬ 
forms. 


RMSE  Center  (m) 

RMSE  Edge  (m) 

Peak  Estimator 

No  Interpolation 

0.200 

0.200 

Linear  Interpolation 

0.200 

0.200 

Spline  Interpolation 

0.000 

0.020 

Pchip  Interpolation 

0.200 

0.200 

Matched  Filter  Time  Domain 

Known  Width 

0.001 

0.310 

EWNA 

0.001 

0.310 

Matched  Filter  Freq  Domain 

Known  Width 

0.225 

0.297 

EWNA 

0.225 

0.297 

Maximum  Likelihood 

Known  Width 

0.003 

0.315 

EWNA 

0.003 

0.315 

Normalized  Matched  Filter 

Known  Width 

0.000 

0.000 

EWNA 

0.000 

0.000 

Figure  5.2  shows  a  comparison  of  the  average  range  found  by  the  different  in¬ 
terpolation  methods  of  the  peak  estimator.  Because  no  noise  was  added,  the  only 
error  in  the  results  is  produced  from  interpolation  or  sampling  error.  Interpolation  of 
the  waveform  was  implemented  using  the  built  in  interpolation  packages  in  Matlab®. 
The  peak  estimator  has  approximately  the  same  average  RMSE  for  both  the  centered 
and  non  centered  waveforms.  The  only  difference  in  error  occurs  when  a  waveform’s 
peak  was  located  outside  of  the  range  gate.  The  peak  estimator  is  able  to  perform 
accurately  in  a  noiseless  environment  using  the  spline  estimator,  but  cannot  account 
for  waveforms  that  have  a  cropped  peak. 
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Waveform  Position 

Figure  5.1:  Average  range  determined  by  matched  filters  (time  domain,  frequency 
domain,  and  normalized)  and  maximum  likelihood  estimators.  Results 
are  from  noiseless  simulation  where  true  location  of  target  was  at  100 
meters.  A  1  value  for  waveform  position  indicates  a  waveform  on  the  far 
right  side  of  range  gate.  A  20  value  for  waveform  position  indicates  a 
waveform  on  the  far  left  side  of  the  range  gate. 
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Figure  5.2:  Average  range  determined  by  peak  estimator  as  waveforms  change  posi¬ 
tion  within  range  gate  in  noiseless  simulation.  True  location  of  target  was 
100  meters.  A  1  value  for  waveform  position  indicates  a  waveform  on  the 
far  right  side  of  range  gate.  A  20  value  for  waveform  position  indicates  a 
waveform  on  the  far  left  side  of  the  range  gate. 
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5.2.2  Results  of  Simulation  With  Noise.  After  the  noiseless  simulation, 
range  estimation  algorithms  were  tested  against  1000  noisy  versions  of  the  20  different 
waveforms  as  they  shifted  position  through  the  range  gate.  The  results  are  compared 
based  on  RMSE  and  standard  deviation  of  the  measurements  found.  An  average 
RMSE  and  standard  deviation  value  of  the  20  waveform  positions  is  also  calculated 
to  compare  the  overall  performance  of  the  algorithms.  In  addition  to  testing  the  range 
estimation  algorithms,  the  three  PWE  estimate  techniques  (APE,  EWA,  and  EWNA) 
were  also  examined.  Table  5.5  lists  the  average  RMSE  and  standard  deviation  values 
of  the  range  estimation  techniques  for  the  simulation  with  noise. 

5.2.2. 1  Peak  Estimator.  Figure  5.3  shows  the  average  range  found 
by  the  peak  estimator  algorithm  as  the  waveforms  shift  position  within  the  range 
gate.  The  RMSE  with  respect  to  waveform  position  is  illustrated  in  Figure  5.4.  Over 
the  center  10  waveforms,  the  peak  estimator  had  the  largest  average  RMSE  of  about 
0.345  m.  This  is  approximately  the  same  as  the  average  RMSE  at  the  edges  of  the 
range  gate,  as  shown  in  Table  5.5.  The  peak  estimator  appears  to  be  unaffected  by 
waveform  position  as  long  as  the  peak  is  contained  in  the  range  gate,  but  it  is  grossly 
affected  by  the  noise  and  level  of  interpolation.  As  shown  in  Figures  5.3  -  5.5  the  raw 
uninterpolated  waveform  does  not  achieve  the  results  of  the  interpolated  waveforms. 
The  cubic  spline  method  of  interpolation  is  the  most  effective  at  reducing  error  caused 
by  interpolation,  even  in  the  presence  of  the  Poisson  noise. 

The  standard  deviation  of  the  peak  estimator  does  not  change  much  as  it  shifts 
within  the  range  gate  and  is  the  highest  value  of  all  methods  in  the  center  of  the  range 
gate.  Figure  5.5  shows  the  standard  deviation  of  the  peak  estimators  with  respect  to 
waveform  position.  The  drop  in  standard  deviation  at  the  edges  is  due  to  the  cropped 
waveforms  that  may  have  not  contained  the  true  peak.  The  smaller  amplitudes  of 
the  cropped  waveforms  produces  less  noise  added  from  the  Poisson  distribution  which 
may  have  contributed  to  less  variance  among  the  measurements  as  the  peak  was  less 
likely  to  shift  position. 
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Table  5.5:  Results  of  simulation  of  range  estimation  when  Poisson  noise  is  added  to 
waveforms.  All  values  have  units  of  meters. 


RMSE  Center 

RMSE  Edge 

Std  Dev  Center 

Std  Dev  Edge 

Peak  Estimator 

No  Interpolation 

0.3638 

0.3611 

0.3635 

0.3458 

Linear  Interpolation 

0.3631 

0.3603 

0.3628 

0.3451 

Spline  Interpolation 

0.3296 

0.3257 

0.3295 

0.3057 

Pchip  Interpolation 

0.3638 

0.3611 

0.3635 

0.3458 

Matched 

Alter  Time  Domain 

Known  Width 

0.0883 

0.3355 

0.0883 

0.0730 

APE 

0.0883 

0.3300 

0.0883 

0.0728 

EWA 

0.0868 

0.3456 

0.0867 

0.0715 

EWNA 

0.0874 

0.3514 

0.0874 

0.0928 

Matched  Filter  Freq  Domain 

Known  Width 

0.3051 

0.3733 

0.3036 

0.2881 

APE 

0.3051 

0.3633 

0.3036 

0.2971 

EWA 

0.3005 

0.3355 

0.2984 

0.2423 

EWNA 

0.2993 

0.3785 

0.2970 

0.3162 

Maximum  Likelihood 

Known  Width 

0.1732 

0.4430 

0.1732 

0.3074 

APE 

0.1732 

0.4450 

0.1732 

0.3065 

EWA 

0.1732 

0.4430 

0.1732 

0.3109 

EWNA 

0.1732 

0.4419 

0.1732 

0.3139 

Normalized  Matched  Filter 

Known  Width 

0.0886 

0.1241 

0.0886 

0.1239 

APE 

0.0886 

0.1291 

0.0886 

0.1206 

EWA 

0.0872 

0.1381 

0.0871 

0.1249 

EWNA 

0.0878 

0.1798 

0.0878 

0.1646 
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Figure  5.3:  Average  range  determined  by  peak  estimator  as  waveforms  change  posi¬ 
tion  within  range  gate.  True  target  location  was  100  meters.  A  1  value 
for  waveform  position  indicates  a  waveform  on  the  far  right  side  of  range 
gate.  A  20  value  for  waveform  position  indicates  a  waveform  on  the  far 
left  side  of  the  range  gate. 
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Figure  5.4:  R.MSE  of  peak  estimator  algorithm  as  waveforms  change  position  within 
range  gate.  A  1  value  for  waveform  position  indicates  a  waveform  on  the 
far  right  side  of  range  gate.  A  20  value  for  waveform  position  indicates  a 
waveform  on  the  far  left  side  of  the  range  gate. 
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Figure  5.5:  Standard  deviation  of  peak  estimator  results  as  waveforms  change  position 
within  range  gate.  A  1  value  for  waveform  position  indicates  a  waveform 
on  the  far  right  side  of  range  gate.  A  20  value  for  waveform  position 
indicates  a  waveform  on  the  far  left  side  of  the  range  gate. 
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5. 2. 2. 2  Matched  Filter.  Based  on  the  results  of  the  simulation  without 
noise,  both  the  matched  filters  in  the  time  and  the  frequency  domains  were  expected 
to  show  a  bias  in  the  range  measurements  at  the  edges  of  the  range  gate.  As  shown  by 
Figures  5.6  the  matched  filter  techniques  underestimate  the  range  when  the  waveform 
is  shifted  left  of  center  and  overestimate  the  range  when  the  waveform  is  shifted  to 
the  right  of  center.  This  is  also  seen  in  the  RMSE  plots  shown  in  Figure  5.7  as  the 
error  drops  dramatically  in  the  center  of  the  range  gate  and  rises  with  the  first  and 
last  3  waveforms.  The  frequency  domain  implementation  of  the  matched  filter  does  a 
better  job  of  adapting  to  the  shifting  waveforms  than  the  time  domain  counterpart  as 
indicated  by  the  lower  RMSE  at  the  edges  of  the  range  gate.  However,  the  matched 
filter  in  the  time  domain  has  a  lower  overall  bias  due  to  its  stronger  performance  with 
the  waveforms  that  are  centered  in  the  range  gate.  In  the  center  of  the  range  gate 
the  matched  filter  time  domain  implementation  had  the  lowest  RMSE  of  the  different 
range  estimation  techniques  tried  with  an  average  RMSE  of  under  approximately 
0.088  m  and  an  average  standard  deviation  over  the  same  interval  of  about  0.073  m. 
The  frequency  domain  implementation  produced  an  error  of  approximately  0.373  m 
over  the  center  samples,  which  is  not  any  better  than  the  peak  estimator. 

The  three  different  PWE  algorithms  were  tested  alongside  a  comparison  to 
using  the  known  standard  deviation  with  both  matched  filter  techniques.  There  was 
not  a  dramatic  difference  in  the  results,  which  could  be  due  to  the  fact  that  timing 
jitter  noise  was  not  simulated  to  cause  waveforms  to  shift.  The  matched  filter  in  the 
frequency  domain  does  appear  to  be  slightly  more  sensitive  to  the  changes  in  the  PWE 
technique,  while  the  time  domain  version  does  not  appear  to  be  too  sensitive  based 
on  the  technique  used.  Table  5.5  lists  the  results  broken  down  by  PWE  technique  for 
comparison. 
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(b)  Matched  Filter  Frequency  Domain 
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(d)  Normalized  Matched  Filter 


Figure  5.6:  Average  range  found  by  matched  filter  and  maximum  likelihood  estima¬ 
tors  in  simulation  with  noise  added.  True  target  location  is  100  meters. 
A  1  value  for  waveform  position  indicates  a  waveform  on  the  far  right  side 
of  range  gate.  A  20  value  for  waveform  position  indicates  a  waveform  on 
the  far  left  side  of  the  range  gate. 
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Figure  5.7:  RMSE  found  by  matched  filter  and  maximum  likelihood  estimators  in 
simulation  with  noise  added.  A  1  value  for  waveform  position  indicates 
a  waveform  on  the  far  right  side  of  range  gate.  A  20  value  for  waveform 
position  indicates  a  waveform  on  the  far  left  side  of  the  range  gate. 
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Figure  5.8:  Standard  deviation  of  range  measurements  found  by  matched  filter  and 
maximum  likelihood  estimators  in  simulation  with  noise  added.  A  1  value 
for  waveform  position  indicates  a  waveform  on  the  far  right  side  of  range 
gate.  A  20  value  for  waveform  position  indicates  a  waveform  on  the  far 
left  side  of  the  range  gate. 
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5. 2. 2. 3  Maximum  Likelihood.  The  maximum  likelihood  estimator  per¬ 
formed  similarly  to  the  matched  filter  in  the  time  domain  with  respect  to  the  RMSE. 
Its  average  error  was  worse  with  an  average  RMSE  of  about  0.173  cm  over  the  center 
10  waveforms.  It  displayed  the  strongest  reaction  to  off  centered  waveforms,  with  a 
an  average  RMSE  of  0.443  m  over  the  10  off  center  waveforms.  The  ML  estimator, 
while  dependent  upon  waveform  models,  was  the  least  sensitive  to  changes  in  PWE 
technique.  It  appears  that  while  the  underlying  model  of  a  Gaussian  pulse  with  Pois¬ 
son  noise  distribution  for  the  waveform  is  important  in  determining  an  accurate  range 
estimate,  the  algorithm  is  not  very  sensitive  to  the  actual  estimates  used  for  the  pulse 
width.  The  degree  to  which  a  pulse  width  estimate  can  be  incorrect  and  still  yield  an 
accurate  result  was  not  explored  any  further. 

5. 2. 2. 4  Normalized  Matched  Filter.  The  normalized  matched  filter  was 
chosen  for  implementation  to  remove  the  bias  in  range  measurements  caused  by  off 
center  waveforms.  Figures  5.6  and  5.7  show  that  the  bias  that  occurs  due  to  waveform 
position  and  unbalanced  energy  levels  is  successfully  removed  from  the  estimations 
when  using  the  normalization  technique  before  correlation.  The  normalized  matched 
filter,  however,  does  not  improve  the  standard  deviation  of  the  measurements  in  the 
center  of  the  range  gate  as  seen  in  5.8.  This  is  likely  due  to  the  mean  value  of  the 
results  increasing  so  any  deviation  from  the  truth  estimate  appears  as  a  larger  error 
in  the  standard  deviation. 

When  using  the  known  value  for  the  pulse  width,  the  normalized  matched  filter 
produces  excellent  results  that  reduce  the  bias  from  measurements  caused  by  waveform 
position.  In  comparison  to  the  time  domain  matched  filter,  the  normalized  matched 
filter  offered  an  improvement  in  average  RMSE  of  0.211  nr  in  the  edge  of  the  range 
gate  when  using  the  known  pulse  width.  However,  when  using  the  alternate  methods 
of  EWA  or  EWNA,  the  bias  is  increased  up  to  0.18  nr.  It  appears  from  the  results  of 
the  simulation  with  noise  added  that  the  normalized  matched  filter  is  very  sensitive  to 
errors  in  the  pulse  width  estimate  and  does  not  perform  much  better  than  the  matched 
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filter  in  the  time  domain  when  the  pulse  width  estimate  is  inaccurate.  The  APE 
algorithm  performed  satisfactorily  because  it  averaged  out  the  noise  before  estimating 
the  pulse  width. 

5.2.3  Overall  Performance  of  Algorithms  in  Simulated  Data.  It  can  be 
seen  from  the  results  in  the  simulation  that  the  strongest  performer  in  reducing  the 
bias  produced  from  random  noise  and  shifted  waveforms  is  the  normalized  matched 
filter.  The  maximum  likelihood  and  time  domain  matched  filter  estimators  also  deliver 
favorable  results,  while  the  peak  estimator  and  matched  filter  in  the  frequency  domain 
produce  the  largest  error. 

5.3  Measured  Data  Results 

The  first  sets  of  data  were  processed  using  an  11  x  11  region  of  pixels  that 
showed  the  brightest  intensity.  The  smaller  region  of  high  intensity  pixels  were  chosen 
because  in  theory  the  stronger  signals  would  produce  better  range  estimates  as  the 
Signal-to-Noise  Ratio  (SNR)  was  increased.  Four  data  sets  were  collected,  and  the 
average  range  differences  were  found  for  each  scenario.  Similar  to  the  simulated  data, 
the  PWE  algorithms  were  tested  alongside  the  range  estimation  techniques.  Data 
was  also  processed  using  an  area  that  contained  approximately  500  pixels  to  see  what 
would  happen  when  a  wider  range  of  signal  strengths  were  used  to  process  the  range 
estimation  algorithms.  The  results  of  the  4  data  sets  were  averaged,  and  the  average 
RMSE,  average  standard  deviation,  and  average  total  error  are  presented  in  Tables 
5.6  and  5.7.  The  results  of  the  individual  data  sets  are  contained  in  Appendix  A. 
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Tabic  5.6:  Results  of  measured  data  experiment  using  121  pixels  to  find  average  range 
difference. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.2053 

0.0675 

0.2225 

Linear  Interpolation 

0.1908 

0.0509 

0.2029 

Spline  Interpolation 

0.1608 

0.0467 

0.1719 

Pchip  Interpolation 

0.2044 

0.0601 

0.2189 

Matched  Filter  Time  Domain 

APE 

0.1011 

0.0568 

0.1189 

EWA 

0.1065 

0.0640 

0.1277 

EWNA 

0.1025 

0.0590 

0.1214 

Matched  Filter  Freq  Domain 

APE 

0.2144 

0.0671 

0.2286 

EWA 

0.2253 

0.0868 

0.2487 

EWNA 

0.2228 

0.0848 

0.2447 

Maximum  Likelihood 

APE 

0.3966 

0.0492 

0.4016 

EWA 

0.3965 

0.0491 

0.4016 

EWNA 

0.3966 

0.0492 

0.4017 

Normalized 

Matched  Filter 

APE 

0.1330 

0.0414 

0.1432 

EWA 

0.1433 

0.0591 

0.1613 

EWNA 

0.1398 

0.0528 

0.1549 
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Tabic  5.7:  Results  of  measured  data  experiment  using  approximately  500  pixels  to 
find  average  range  difference. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.2195 

0.0540 

0.2263 

Linear  Interpolation 

0.2017 

0.0411 

0.2061 

Spline  Interpolation 

0.1755 

0.0333 

0.1791 

Pchip  Interpolation 

0.2138 

0.0421 

0.2183 

Matched  Filter  Time  Domain 

APE 

0.0932 

0.0648 

0.1175 

EWA 

0.0949 

0.0669 

0.1201 

EWNA 

0.0912 

0.0625 

0.1145 

Matched  Filter  Freq  Domain 

APE 

0.1471 

0.0644 

0.1632 

EWA 

0.1666 

0.0880 

0.1927 

EWNA 

0.1569 

0.0876 

0.1829 

Maximum  Likelihood 

APE 

0.4677 

0.0472 

0.4713 

EWA 

0.4676 

0.0472 

0.4712 

EWNA 

0.4676 

0.0472 

0.4712 

Normalized 

Matched  Filter 

APE 

0.0921 

0.0505 

0.1090 

EWA 

0.1013 

0.0620 

0.1231 

EWNA 

0.1038 

0.0578 

0.1235 

66 


5.3.1  Peak  Estimator.  Similar  to  the  simulation,  the  peak  estimator  was 
applied  to  waveforms  using  four  different  interpolation  strategies.  In  every  set  of  data, 
some  level  of  interpolation  reduced  the  RMSE.  The  averaged  RMSE  for  uninterpolated 
waveforms  using  the  peak  estimator  in  a  121  pixel  region  was  .2053  m  and  the  least 
RMSE  was  produced  by  the  spline  interpolated  waveform  at  .1608  m,  nearly  a  4  cm 
improvement  in  bias  reduction.  In  the  regions  of  500  pixels  the  RMSE  was  slightly 
increased  for  each  method,  with  the  uninterpolated  waveforms  producing  an  RMSE 
of  .2195  m.  The  best  interpolation  method  in  the  500  pixel  region  was  still  the 
spline  method  with  an  increased  RMSE  of  .1755.  The  lower  SNR  of  the  decreased 
intensity  signals  could  explain  for  more  shifted  peaks  and  greater  interpolation  error 
in  the  interpolated  signals  when  using  a  larger  area  of  pixels  for  the  range  estimation 
measurements. 

The  peak  estimator  methods  are  exceptionally  fast  when  implemented  in  Matlab® 
using  a  2.49  GHz  processor.  Without  interpolation,  the  peak  estimator  can  process 
121  pixels  in  about  0.25  seconds.  The  peak  estimator  using  spline  interpolation  is 
about  twice  as  slow,  but  still  able  to  process  121  pixels  in  about  .59  seconds.  In¬ 
terpolation  makes  the  algorithm  slower  than  the  frequency  domain  version  of  the 
matched  filter,  but  still  about  8  times  faster  than  the  slowest  algorithm,  the  normal¬ 
ized  matched  filter.  In  spite  of  its  speed,  the  bias  in  a  peak  estimator  is  still  in  the 
range  of  16-21  cm.  The  tradeoff  in  the  peak  estimator  lies  in  the  balance  between 
speed  and  accuracy.  Table  5.8  shows  the  results  of  the  other  algorithms  in  terms  of 
processing  time. 


Table  5.8:  Algorithm  processing  times  using  a  computer  with  a  2.49  GHz  processor. 


Algorithm 

121  pixels  (sec) 

Full  128  x  128  Array  (min) 

Peak  Estimator  (no  interp.) 

0.0363 

0.0819 

Peak  Estimator  (spline  interp.) 

0.2541 

0.5734 

Matched  Filter  Freq.  Domain 

0.1452 

0.3277 

Matched  Filter  Time  Domain 

1.2015 

2.7034 

Maximum  Likelihood 

0.7151 

1.6111 

Normalized  Matched  Filter 

2.2554 

5.0790 
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5.3.2  Matched  Filter.  The  matched  filter  was  implemented  in  three  different 
ways.  The  time  domain,  frequency  domain,  and  normalized  versions  each  had  unique 
results  involving  the  PWE  techniques  applied  to  each  algorithm. 

5.3.2. 1  Time  Domain.  In  contrast  to  the  simulations,  the  unnormal¬ 
ized  matched  filter  implemented  in  the  time  domain  experiment  proved  to  yield  the 
most  positive  results  out  of  all  the  algorithms  in  terms  of  bias  on  average.  There  is 
still  a  bias  in  the  measurements  most  likely  due  to  inaccuracies  in  the  reference  model 
and  waveforms  that  are  not  centered  in  the  range  gate.  The  matched  filter  in  the 
time  domain  was  able  to  accurately  distinguish  ranges  within  approximately  10  cm 
when  121  pixels  were  averaged  and  within  about  6  cm  when  500  pixels  were  used. 

The  time  domain  algorithm  implementation  produced  varying  results  depending 
on  the  PWE  technique  applied.  In  the  121  pixel  trials,  the  APE  algorithm  produced 
the  least  total  error  of  .1189  m.  In  the  the  500  pixel  trials,  the  EWA  algorithm 
produced  the  least  total  error  of  .1145  m.  With  a  smaller  area  of  pixels,  there  is  less 
likely  a  chance  for  timing  jitter  or  other  noises  to  shift  the  pulses,  so  using  an  averaged 
waveform  to  calculate  the  pulse  width  produces  a  lower  error.  However,  in  the  larger 
region,  it  is  more  sensible  to  calculate  the  pulse  width  for  each  waveform  and  then 
average  as  there  is  more  likely  a  chance  for  the  waveforms  to  shift  position. 

5. 3. 2. 2  Frequency  Domain.  The  frequency  domain  matched  filter 
behaved  as  expected  based  on  the  simulated  results.  It  exhibited  a  large  degree  of 
standard  deviation  among  the  measurements  and  showed  a  large  bias  in  the  RAISE. 
Similar  to  the  time  domain  implementation,  the  results  of  the  frequency  domain 
implementation  improved  by  using  a  larger  pixel  area.  The  APE  method  produced 
the  least  total  error  in  both  the  121  and  500  pixel  regions  with  values  of  .2286  m  and 
.1632  m  respectively. 

5. 3. 2. 3  Normalized.  The  normalized  matched  filter  had  high  expecta¬ 
tions  for  removing  bias  in  the  range  measurements  of  waveforms  that  were  off  center. 
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However,  it  did  not  reduce  the  RMSE  as  it  did  in  simulations  and  did  not  perform 
as  well  as  the  un-normalized  matched  filter.  The  average  RMSE  of  the  normalized 
matched  filter  was  at  best  .133  nr  when  using  the  APE  algorithm  and  had  a  relatively 
low  standard  deviation  of  .0414  m  in  the  121  pixel  data  set.  When  500  pixels  were 
used  in  the  range  estimation  the  average  RMSE  dropped  about  4  cm  to  .0921  m  with 
a  standard  deviation  of  about  .0505  nr  as  seen  in  Table  5.7.  The  unexpected  poor 
performance  in  the  measured  data  is  due  to  the  normalized  matched  filter’s  sensitiv¬ 
ity  to  inaccuracies  in  the  reference  waveform.  As  shown  in  the  simulations,  when  the 
pulse  width  estimate  is  off,  the  normalized  matched  filter  performs  about  as  well  as 
the  matched  filter  in  the  time  domain. 

5.3.3  Maximum  Likelihood.  The  maximum  likelihood  estimator  provided 
the  results  largest  in  contrast  to  the  simulated  results.  Instead  of  performing  com¬ 
parable  to  the  matched  filter  implementation,  it  showed  a  large  degree  of  total  error 
in  measurements.  It  consistently  underestimated  the  range.  Unlike  the  other  range 
estimation  algorithms,  the  ML  estimator  did  not  improve  when  the  number  of  pixels 
used  in  the  estimation  was  increased  from  121  to  500.  Total  error  increased  about  7 
cm  from  .4016  nr  in  the  121  pixel  region  to  .4713  nr  in  the  500  pixel  region.  In  con¬ 
trast  to  the  matched  filter,  the  maximum  likelihood  algorithm  does  not  seem  to  be 
affected  much  by  the  PWE  technique  chosen.  Referencing  Tables  5.6  and  5.7,  the  ML 
estimator  delivered  approximately  the  same  results  regardless  of  the  PWE  method. 
The  ML  estimator  appears  to  be  sensitive  to  changes  in  the  underlying  model,  but 
the  pulse  width  estimate  used  does  not  have  as  strong  as  an  effect  on  the  error.  As 
more  pixels  are  used  in  the  range  estimate,  there  is  a  stronger  chance  of  using  pixels 
that  deviate  from  the  original  model.  It  would  appear  from  the  results  the  model 
itself  for  the  ML  estimator  (Gaussian  wave  with  Poisson  noise)  would  need  refining 
to  improve  the  estimator. 

5.3.4  Clock  Cycle  Observations  in  ASC  3-D  Flash  LADAR  Camera.  If  the 
average  range  difference  results  for  each  cube  of  data  are  plotted  in  time,  then  it 
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Figure  5.9:  Range  estimate  techniques  with  respect  to  clock  cycle. 

is  possible  to  track  the  clock  cycle  of  the  camera  as  each  laser  pulse  is  fired.  The 
results  can  provide  insight  to  if  there  are  any  other  additional  noise  sources  in  the 
camera  based  on  timing  or  other  unknown  factors.  Figure  5.9  shows  the  results  of 
one  data  set  when  the  true  range  difference  was  1.19  meters.  The  values  along  the 
y-axis  indicate  the  average  range  difference  found  in  each  cube  of  data  produced  by 
a  return  pulse,  and  the  x-axis  indicates  the  pulse  or  data  cube  number.  As  seen  in 
Figure  5.9  there  is  a  fluctuation  in  values  that  could  be  dependent  on  internal  clock  of 
the  LADAR  system.  Figure  5.10  shows  the  average  error  (range  difference  -  truth)  in 
each  algorithm  with  respect  to  the  clock  cycle.  A  designed  test  with  more  pulses  fired 
would  be  necessary  to  determine  if  the  values  would  stabilize  over  time  or  continue 
to  fluctuate. 
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Figure  5.10:  Range  error  with  respect  to  clock  cycle. 
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VI.  Conclusion 


6.1  Conclusions  from  Results 

Based  on  the  experiments  conducted  in  the  scope  this  research,  several  key 
conclusions  can  be  drawn  from  the  results.  As  observed  in  other  projects  involving 
the  ASC  3-D  Flash  LADAR  camera,  the  noise  contained  in  individual  measurements 
makes  absolute  range  estimates  in  a  single  measurement  unreliable  [7].  Any  usable 
range  measurement  where  the  precision  is  desired  to  be  within  several  centimeters 
must  be  averaged  over  multiple  frames  to  eliminate  the  effects  of  the  shot  noise. 

The  peak  estimator  is  a  quick  and  effective  tool  for  estimating  ranges  provided 
there  are  several  hundred  pixels  of  information  and  frames  to  average  over.  The  peak 
estimator  would  probably  be  best  for  averaging  out  the  noise  to  estimate  a  range 
to  large  flat  surface  target.  It  would  most  likely  not  be  able  to  distinguish  features 
that  contained  a  great  deal  of  detail  unless  there  were  numerous  frames  collected  to 
average  over. 

The  model  used  for  the  maximum  likelihood  estimator  is  most  likely  flawed 
based  on  the  strong  bias  error  shown  in  the  results  of  the  measured  data  experiment. 
Based  on  the  simulations,  it  has  been  shown  to  be  effective  when  the  model  is  prop¬ 
erly  calibrated,  but  the  measured  data  results  show  the  potential  for  high  error  when 
using  the  wrong  combination  of  reference  wave  and  noise  distribution.  In  the  sim¬ 
ulation,  the  only  sources  of  noise  were  from  simulated  background  and  shot  noise. 
The  measured  data  could  possibly  also  contain  a  source  of  Gaussian  noise  from  the 
electronics  involved  that  was  not  properly  accounted  for  when  designed  the  maximum 
likelihood  algorithm.  Further  research  could  explore  which  noise  type  (Poisson  versus 
Gaussian)  has  the  greatest  effect  on  the  signal.  Because  the  probability  distribution 
for  the  waveform  and  noise  will  always  be  somewhat  of  an  estimate,  it  may  never 
prove  to  be  a  reliable  source  for  range  estimation. 

The  matched  filters,  like  the  maximum  likelihood  estimator,  show  a  strong  po¬ 
tential  for  excellent  range  estimation  in  simulations.  However,  their  dependence  on 
the  accuracy  of  a  reference  waveform  limit  their  potential  in  measured  data.  While 
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the  normalization  technique  presented  in  the  research  can  eliminate  the  bias  produced 
in  partial  or  uncentered  waveforms,  it  greatly  increases  the  sensitivity  of  the  estimator 
with  respect  to  its  reference  waveform.  The  unnormalized  matched  filter  technique 
involves  less  mathematical  operations  and  will  therefore  perform  faster.  By  not  nor¬ 
malizing  the  waveforms,  the  estimation  technique  is  vulnerable  to  a  bias  error  from 
waveforms  that  are  on  the  edges  of  the  range  gate,  but  it  is  not  as  sensitive  to  the 
errors  in  pulse  width  estimation.  The  matched  filter  implemented  in  the  time  domain 
is  a  stronger  performer  in  terms  of  both  bias  and  variance  of  range  measurements 
when  compared  to  the  frequency  domain  implementation.  To  account  for  the  speed 
difference  that  is  often  favored  by  the  frequency  domain  implementation,  one  could 
implement  look  up  tables  and  preprogrammed  values  for  reference  waveforms  or  pulse 
width  estimates. 

The  choice  in  algorithm  would  rely  mainly  on  an  application’s  exact  require¬ 
ments  for  speed  and  accuracy.  Algorithms  such  as  the  peak  estimator  or  frequency 
domain  matched  filter  display  a  lower  degree  of  accuracy,  but  deliver  strong  perfor¬ 
mance  in  the  speed  category.  A  time  domain  implementation  of  the  matched  filter 
may  improve  range  estimation  accuracy,  but  will  suffer  from  increased  processing 
time.  The  balance  in  trade  off  between  speed  and  accuracy  will  have  to  ultimately  be 
determined  by  user  requirements. 

6.2  Recommendations  for  Future  Research 

Several  options  exist  for  expanding  the  research  performed  during  the  scope  of 
this  thesis.  One  could  spend  more  time  characterizing  the  pulse  shapes  produced 
by  the  laser  in  order  to  form  a  more  robust  waveform  model.  This  could  be  in 
terms  of  pulse  width  shape  for  use  in  the  matched  filters  or  in  terms  of  probability 
distribution  to  use  with  a  maximum  likelihood  algorithm.  Another  option  could 
involve  finding  a  faster  and  more  accurate  method  for  estimating  the  pulse  width 
used  for  the  developing  the  waveform  model.  The  effects  of  gain  variation  have  been 
studied  with  the  older  camera  model,  but  it  has  not  yet  been  documented  with  the 
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new  camera  to  verify  if  any  conclusions  can  be  drawn  regarding  the  effects  of  non 
linear  gain  in  the  pixels  of  the  new  camera  that  was  used  to  collect  data  for  this 
research.  Based  on  the  simulations  that  show  the  strong  potential  of  a  maximum 
likelihood  estimator  with  an  accurate  model,  it  would  be  useful  to  develop  other 
maximum  likelihood  estimators  with  different  combinations  of  pulse  model  and  noise 
distribution  to  test  in  a  controlled  measured  data  experiment  to  see  which  performed 
the  best  and  which  was  the  most  sensitive  to  changes  in  pulse  width  estimation  errors. 
The  same  process  could  be  implemented  to  attempt  to  design  a  correlation  technique 
that  removes  the  bias  like  the  normalization  process,  but  also  minimizes  the  overall 
sensitivity  to  slight  changes  in  the  reference  model. 


74 


Appendix  A.  Results  From  Individual  Data  Sets 

A.l  Results  from  Sular9ftl .SEQ 

Table  A.l  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sular9ftl.SEQ  when  121  pixels  were  used  in  the  range  measurements. 

Table  A.l:  Results  from  Sular9ftl.SEQ  using  121  pixels  to  find  average  range  differ¬ 
ence. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.2144 

0.0114 

0.2147 

Linear  Interpolation 

0.1921 

0.0122 

0.1925 

Spline  Interpolation 

0.1463 

0.0135 

0.1469 

Pchip  Interpolation 

0.2365 

0.0245 

0.2377 

Matched  Filter  Time  Domain 

APE 

0.0837 

0.0242 

0.0871 

EWA 

0.0843 

0.0235 

0.0875 

EWNA 

0.0841 

0.0229 

0.0872 

Matched  Filter  Freq  Domain 

APE 

0.2403 

0.0152 

0.2407 

EWA 

0.2402 

0.0149 

0.2406 

EWNA 

0.2394 

0.0153 

0.2398 

Maximum  Likelihood 

APE 

0.0794 

0.0253 

0.0833 

EWA 

0.0792 

0.0251 

0.0831 

EWNA 

0.0793 

0.0252 

0.0832 

Normalized 

Matched  Filter 

APE 

0.1216 

0.0147 

0.1225 

EWA 

0.1194 

0.0161 

0.1205 

EWNA 

0.1226 

0.0159 

0.1236 
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Table  A. 2  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sular9ftl.SEQ  when  approximately  500  pixels  were  used  in  the  range  measurements. 

Table  A. 2:  Results  from  Sular9ftl.SEQ  using  approximately  500  pixels  to  find  average 
range  difference. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.1377 

0.0328 

0.1415 

Linear  Interpolation 

0.0967 

0.0305 

0.1014 

Spline  Interpolation 

0.0644 

0.0201 

0.0675 

Pchip  Interpolation 

0.1344 

0.0216 

0.1361 

Matched  Filter  Time  Domain 

APE 

0.0344 

0.0330 

0.0477 

EWA 

0.0338 

0.0322 

0.0467 

EWNA 

0.0349 

0.0317 

0.0472 

Matched  Filter  Freq  Domain 

APE 

0.1486 

0.0437 

0.1548 

EWA 

0.1485 

0.0449 

0.1551 

EWNA 

0.1423 

0.0480 

0.1502 

Maximum  Likelihood 

APE 

0.1339 

0.0259 

0.1363 

EWA 

0.1337 

0.0258 

0.1362 

EWNA 

0.1336 

0.0258 

0.1361 

Normalized 

Matched  Filter 

APE 

0.1216 

0.0147 

0.0588 

EWA 

0.1194 

0.0161 

0.0583 

EWNA 

0.1226 

0.0159 

0.0634 
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A. 2  Results  from  Sular9ft2.SEQ 


Table  A. 3  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sular9ft2.SEQ  when  121  pixels  were  used  in  the  range  measurements. 

Table  A. 3:  Results  from  Sular9ft2.SEQ  using  121  pixels  to  find  average  range  differ¬ 
ence. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.2709 

0.0403 

0.2739 

Linear  Interpolation 

0.2830 

0.0288 

0.2845 

Spline  Interpolation 

0.1997 

0.0232 

0.2011 

Pchip  Interpolation 

0.2892 

0.0325 

0.2910 

Matched  Filter  Time  Domain 

APE 

0.0312 

0.0238 

0.0392 

EWA 

0.0307 

0.0230 

0.0383 

EWNA 

0.0309 

0.0227 

0.0834 

Matched  Filter  Freq  Domain 

APE 

0.2979 

0.0743 

0.3070 

EWA 

0.2986 

0.0773 

0.3085 

EWNA 

0.3095 

0.0886 

0.3219 

Maximum  Likelihood 

APE 

0.1975 

0.0230 

0.1989 

EWA 

0.1975 

0.0230 

0.1989 

EWNA 

0.1978 

0.0232 

0.1991 

Normalized 

Matched  Filter 

APE 

0.1499 

0.0151 

0.1507 

EWA 

0.1479 

0.0185 

0.1491 

EWNA 

0.1509 

0.0190 

0.1521 
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Table  A. 4  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sular9ft2.SEQ  when  approximately  500  pixels  were  used  in  the  range  measurements. 

Table  A. 4:  Results  from  Sular9ft2.SEQ  using  approximately  500  pixels  to  find  average 
range  difference. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.1736 

0.0351 

0.1771 

Linear  Interpolation 

0.1710 

0.0178 

0.1719 

Spline  Interpolation 

0.1099 

0.0270 

0.1132 

Pchip  Interpolation 

0.1815 

0.0199 

0.1826 

Matched  Filter  Time  Domain 

APE 

0.0625 

0.0341 

0.0713 

EWA 

0.0613 

0.0331 

0.0697 

EWNA 

0.0579 

0.0323 

0.0663 

Matched  Filter  Freq  Domain 

APE 

0.1729 

0.0573 

0.1822 

EWA 

0.1734 

0.0611 

0.1838 

EWNA 

0.1590 

0.0701 

0.1738 

Maximum  Likelihood 

APE 

0.2502 

0.0224 

0.2512 

EWA 

0.2501 

0.0224 

0.2511 

EWNA 

0.2500 

0.0224 

0.2510 

Normalized 

Matched  Filter 

APE 

0.0760 

0.0253 

0.0800 

EWA 

0.0754 

0.0280 

0.0804 

EWNA 

0.0889 

0.0246 

0.0922 
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A. 3  Results  from  Sularl9ftl.SEQ 


Table  A. 5  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sularl9ftl.SEQ  when  121  pixels  were  used  in  the  range  measurements. 

Table  A. 5:  Results  from  Sularl9ftl.SEQ  using  121  pixels  to  find  average  range  differ¬ 
ence. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.1126 

0.0594 

0.1273 

Linear  Interpolation 

0.0967 

0.0278 

0.1006 

Spline  Interpolation 

0.0919 

0.0130 

0.0929 

Pchip  Interpolation 

0.0938 

0.0388 

0.1015 

Matched  Filter  Time  Domain 

APE 

0.0895 

0.0159 

0.0910 

EWA 

0.0879 

0.0160 

0.0894 

EWNA 

0.0876 

0.0160 

0.0890 

Matched  Filter  Freq  Domain 

APE 

0.1516 

0.0706 

0.1672 

EWA 

0.1458 

0.0741 

0.1635 

EWNA 

0.1413 

0.0757 

0.1602 

Maximum  Likelihood 

APE 

0.7152 

0.0149 

0.7154 

EWA 

0.7150 

0.0149 

0.7151 

EWNA 

0.7150 

0.0150 

0.7151 

Normalized 

Matched  Filter 

APE 

0.0896 

0.0159 

0.0910 

EWA 

0.0880 

0.0160 

0.0894 

EWNA 

0.0875 

0.0160 

0.0889 
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Table  A. 6  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sularl9ftl.SEQ  when  approximately  500  pixels  were  used  in  the  range  measurements. 

Table  A. 6:  Results  from  Sularl9ftl.SEQ  using  approximately  500  pixels  to  find  aver¬ 
age  range  difference. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.1493 

0.0549 

0.1590 

Linear  Interpolation 

0.1232 

0.0283 

0.1264 

Spline  Interpolation 

0.1173 

0.0042 

0.1174 

Pchip  Interpolation 

0.1251 

0.0385 

0.1308 

Matched  Filter  Time  Domain 

APE 

0.0885 

0.0090 

0.0900 

EWA 

0.0876 

0.0090 

0.0880 

EWNA 

0.0900 

0.0090 

0.0903 

Matched  Filter  Freq  Domain 

APE 

0.1664 

0.0609 

0.1772 

EWA 

0.1626 

0.0650 

0.1752 

EWNA 

0.1608 

0.0684 

0.1747 

Maximum  Likelihood 

APE 

0.7738 

0.0156 

0.7740 

EWA 

0.7735 

0.0157 

0.7737 

EWNA 

0.7736 

0.0157 

0.7738 

Normalized 

Matched  Filter 

APE 

0.0886 

0.0089 

0.0890 

EWA 

0.0876 

0.0089 

0.0880 

EWNA 

0.0898 

0.0089 

0.0903 
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A. 4  Results  from  Sularl9ft2.SEQ 


Table  A.  7  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sularl9ft2.SEQ  when  121  pixels  were  used  in  the  range  measurements. 

Table  A. 7:  Results  from  Sularl9ft2.SEQ  using  121  pixels  to  find  average  range  differ¬ 
ence. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.2235 

0.1588 

0.2742 

Linear  Interpolation 

0.1914 

0.1348 

0.2341 

Spline  Interpolation 

0.2053 

0.1371 

0.2470 

Pchip  Interpolation 

0.1983 

0.1447 

0.2455 

Matched  Filter  Time  Domain 

APE 

0.2000 

0.1634 

0.2583 

EWA 

0.2231 

0.1936 

0.2954 

EWNA 

0.2076 

0.1743 

0.2711 

Matched  Filter  Freq  Domain 

APE 

0.1678 

0.1081 

0.1996 

EWA 

0.2167 

0.1810 

0.2823 

EWNA 

0.2012 

0.1594 

0.2567 

Maximum  Likelihood 

APE 

0.5943 

0.1330 

0.6090 

EWA 

0.5944 

0.1335 

0.6092 

EWNA 

0.5944 

0.1333 

0.6093 

Normalized 

Matched  Filter 

APE 

0.1708 

0.1197 

0.2086 

EWA 

0.2179 

0.1858 

0.2864 

EWNA 

0.1983 

0.1604 

0.2551 
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Table  A. 8  lists  the  average  RMSE,  standard  deviation,  and  total  error  from 
Sularl9ft2.SEQ  when  approximately  500  pixels  were  used  in  the  range  measurements. 

Table  A. 8:  Results  from  Sularl9ft2.SEQ  using  approximately  500  pixels  to  find  aver¬ 
age  range  difference. 


Avg  RMSE  (m) 

Avg  Std  Dev  (m) 

Avg  Total  Error  (m) 

Peak  Estimator 

No  Interpolation 

0.4173 

0.0934 

0.4276 

Linear  Interpolation 

0.4157 

0.0876 

0.4249 

Spline  Interpolation 

0.4104 

0.0820 

0.4185 

Pchip  Interpolation 

0.4143 

0.0883 

0.4236 

Matched  Filter  Time  Domain 

APE 

0.1874 

0.1833 

0.2621 

EWA 

0.1971 

0.1935 

0.2762 

EWNA 

0.1821 

0.1772 

0.2541 

Matched  Filter  Freq  Domain 

APE 

0.1004 

0.0957 

0.1387 

EWA 

0.1819 

0.1809 

0.2566 

EWNA 

0.1653 

0.1639 

0.2328 

Maximum  Likelihood 

APE 

0.7129 

0.1248 

0.7237 

EWA 

0.7129 

0.1249 

0.7238 

EWNA 

0.7129 

0.1247 

0.7238 

Normalized 

Matched  Filter 

APE 

0.1503 

0.1436 

0.2079 

EWA 

0.1899 

0.1858 

0.2657 

EWNA 

0.1783 

0.1728 

0.2483 
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